Methods and apparatus for receiving and storing in a camera a user controllable setting that is used to control composite image generation performed after image capture

ABSTRACT

A camera device including multiple optical chains receives user input indicating one or more user selectable control option setting(s), e.g., a user selected depth of field control option setting and an aspect ratio control option setting. The indicated control option settings are stored in memory in the camera device. Multiple optical chains of the camera device capture an image during an image capture time interval. A set of captured images are stored in a file along with metadata including the user indicated control option settings. A composite image is generated from a set of captured images in accordance with the stored indicated control option settings, e.g., generating a composite image with a user selected depth of field and/or a user selected aspect ratio. In some embodiments, at least some of the user selected control option settings are not used during image capture operations but are used in subsequent image processing.

FIELD

The present application relates to camera devices and, moreparticularly, methods and apparatus for receiving, storing and/or usinguser selectable image control option settings and/or user selectablecamera control option settings.

BACKGROUND

At the time of image capture a photographer normally considers the sceneand selects a portion of interest to be captured. The photographer mayalso select on or more camera settings which normally control the cameraand image capture process. For example, in a conventional camera anaperture may alter a physical aperture used to capture an image.

Cameras which include multiple optical chains, e.g., camera modules, arenow becoming available. As with more conventional cameras, a user maytake the time to select and scene area of interest and set one or morecamera settings which can affect image capture. In such cameras, thepreview image may be and sometimes is captured by a single cameramodule.

Images captured by a digital camera including multiple optical chainsmay be combined as part of a post capture processing operation togenerate a composite image. Various manual editing options available toan operator of a post capture image processing system allow the operatorto view the effects of different settings on the composite image.However, this post capture editing process may be done by a differentindividual than the photographer who took the picture and may lackinsights into the importance of particular aspects of the scene or theemotional significance of certain portions of the scene, e.g., awedding, since the person doing the post capture editing might not havebeen present at the event and may be unaware of items that wereindicated by the participant to be particularly significant to thephotographer.

Thus since the person controlling the post-capture processing may be adifferent person from the photographer, and he or she may lack theunique perspective on what the photographer considered to be settingsused during post capture image processing which the photographer thoughtwould produce the best composite images. For example, a particulardesired depth of field setting or a particular desired aspect ratiosetting, for the effect that the photographer was trying to achieve. Inaddition, even if a photographer is the same person controlling the postcapture image processing, there may be a large time gap between when theimages are captured and when the post-capture composite images aregenerated and/or the photographer may have taken a very large number ofimages, so that the photographer does not remember his/her originalintended settings and/or desired effect to be achieved for eachcomposite image to be generated.

Furthermore, the post capture processing system may lack informationabout the actual camera settings which were used to capture the originalimages since the images may include only pixel values and notinformation about the camera setting used to capture the imagesrepresented by the pixel values.

Based on the above discussion, it would be beneficial if methods andapparatus were available to allow a user of a camera apparatus, e.g.,the photographer, to input and/or store one or more image control optionsettings prior to capturing images or at the time of image capture sothat the photographer's original intended preferences would be readilyavailable for use during post image capture processing even if thesettings and/or preference information do not affect the image captureoperation.

SUMMARY

In various embodiments a camera user enters one or more image capturesettings. Some of the settings, such as exposure time, may control imagecapture while other setting may be used as recommendations orphotographer preference information which does not control image capturebut how images are processed to generate a composite image after theinitial image capture operation. For example, while aspect ratio, depthof field (DOE) and/or aperture settings may not affect image captureuser settings maybe input and stored prior to or at the time of imagecapture. Such settings can and sometimes are used to control postcapture image processing to generate a composite image with an affectthat simulates what the user would expect if the specified setting wereimplemented in a conventional camera as part of the image captureoperation. In addition, information about the actual camera settingswhich did affect image capture can be and sometimes are stored with thecaptured images along with the user setting recommendations. Such camerasetting information which might not be available in conventional systemscan be and sometimes is used to control post capture image processing.For example exposure setting and/or knowledge that a flash was usedduring image capture can be and sometimes is taken into considerationwhen implementing image post processing.

Since photographer preferences for various settings which can be used tocontrol post image capture image processing can be received and storedprior to or at the time of taking a photograph when the user is mostfamiliar with the scene and the image to be captured, considerable timecan be saved since the user need not enter such information at the timeof post capture image processing.

Depth of field (DOF) and aspect ratio settings may be considered userpreference settings since these settings in some embodiments do notaffect image capture, and the output based on the setting is achievedvia post image capture processing in the camera or in a separateprocessing system, e.g., personal computer or online computer system.

A user desired depth of field setting can be entered in a variety ofways. In some embodiments a DOF setting is entered using a sliderincluding a position indicator, e.g., a large white dot, which can beslid along a predetermined path or using another input option forcontrolling a depth of field setting. In other embodiments the user canindicate a range of depths which are used to indicate a DOF setting bytouching objects at different depths in a preview image to indicate thatall user selected objects which were touched should be in focus.

In some embodiments the depth of field setting affects the range ofdistances at which an image being generated from multiple capturedimages is to be in focus. In one slider embodiment, the depth of fieldslider includes a series of small dots with the dots going from brightto dark on one side of the position indicator with dot brightness beingused to indicate the current setting and/or the direction in which theslider was most recently moved and/or is being moved. The user selecteddepth of field setting is stored in metadata and does not affect actualimage capture or control of images. Based on user input post-captureprocessing is implemented to achieve the user indicated desired depth offield. Thus images are captured, the user indicated desired depth offield information, which is in essence preference information providedat or before image capture time regarding the DOF, is stored in a filewith one or more of the captured images and used to control post captureimage processing. The images are processed, and then the user isdisplayed the resulting image. The processing may be performed by theprocessor in the camera, but in some embodiments the processing isperformed by a computer system, e.g., using cloud based or PC basedsoftware, with the resulting image then being stored, transmitted and/ordisplayed, e.g., on the display of the camera device or another device.

A user desired aspect ratio setting can be input or selected by a userof a camera prior to image capture. The input user desired aspect ratiosetting is stored in metadata to be utilized in post image captureprocessing operations, e.g., performing a cropping operation, inaccordance with the user desired aspect ratio on a composite imagegenerated from a plurality of captured images corresponding to differentoptical chains in the camera device. In various embodiments, the userdesired aspect ratio is not utilized in actual image capture operations,but is utilized in post image capture processing operation, e.g., by thecamera or by a computer system external to the camera.

In some embodiments, additional user selectable image control optionsetting(s), e.g., an F-number setting, and/or user selectable cameracontrol setting(s), e.g., a flash status settings, an ISO setting, anexposure duration setting, etc., are received prior to image capture andstored in metadata along with captured images to be available to be usedin post-capture image processing.

In some embodiments the user settings which are intended to control postimage capture operations rather than image capture are stored in a filealong with the image or images captured by the camera device. The userpreference information, e.g., DOF, Aspect Ratio, etc, maybe stored asmeta data along with camera settings, such as exposure settings, whichaffected the actual image capture process. The setting and preferenceinformation can be and sometimes is included in a file with capturedimage data, e.g., pixel values but also maybe and sometimes are storedseparately with an indication as to the image or images to which thesetting and/or post capture processing preference information applies.Since such information is collected at the time of image capture orimmediately before image capture and stored to facilitate post imagecapture processing, the captured images can be combined based on theentered information without having to seek such input from a user againduring the post capture image processing operation. However, since thecaptured images are preserved, such settings can be altered should auser choose to do so after image capture allowing for a great deal offlexibility with regard to post image capture image processing.

An exemplary method, in accordance with some embodiments, comprises:receiving, at a camera including multiple optical chains, user inputindicating a user selectable image control option setting; storing thereceived user selectable image control option setting in a memory ofsaid camera; operating multiple optical chains to capture images afterreceipt of the user selectable image control option setting; andgenerating from the images captured by the multiple optical chains acomposite image in accordance with the stored user selectable imagecontrol option setting. In some embodiments, the user selectable controlimage option setting is one of: a depth of field setting, an aspectratio setting, and an f-setting.

An exemplary camera apparatus, in accordance with some embodiments,comprises: a plurality of optical chains; memory; a processor; and auser input device configured to receive user input indicating a userselectable image control option setting. In some such embodiments, theprocessor is configured to: store the received user selectable imagecontrol option setting in said memory of said camera apparatus; operatesaid plurality of optical chains to capture images after receipt of theuser selectable image control option setting; and generate from theimages captured by the plurality of optical chains a composite image inaccordance with the stored user selectable image control option setting.

Numerous additional features, benefits and embodiments are described inthe detailed description which follows.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a block diagram of an exemplary apparatus, e.g., a cameradevice, implemented in accordance with one embodiment of the presentinvention.

FIG. 2 illustrates a frontal view of an apparatus implemented inaccordance with an exemplary embodiment which incorporates multipleoptical chains, e.g., camera modules, in accordance with the presentinvention with lenses which are viewable from the front of the camera.

FIG. 3, which is a side view of the exemplary apparatus of FIG. 2,illustrates further details of the exemplary apparatus.

FIG. 4 illustrates a camera device implemented in accordance withanother embodiment.

FIG. 5 illustrates the optical chains of the camera device shown in FIG.4, as implemented in one particular exemplary embodiment, in greaterdetail.

FIG. 6 shows an exemplary user interface screen displayed to the userupon switching the camera device on in a normal mode of operation, inaccordance with one embodiment.

FIG. 7 shows an exemplary user interface screen with a user's fingertapping or being swiped on an area of the display screen.

FIG. 8 shows an exemplary user interface screen displayed to the userupon the camera detecting user input, e.g., finger tap, on the displayscreen over the area shown in FIG. 7.

FIG. 9 illustrates a drawing showing a display screen with the samecontrol options as shown in FIG. 8 but with a user's finger beingadditionally shown tapping/pressing the auto mode icon on the displayscreen.

FIG. 10 illustrates a user interface screen displayed subsequent to theselection of the auto control mode option by the user shown in FIG. 9.

FIG. 11 illustrates a drawing showing a user interface screen with thesame control options as shown in FIG. 8 but with a user's finger beingadditionally shown tapping/pressing the manual mode icon on the displayscreen.

FIG. 12 illustrates a user interface screen displayed subsequent to theselection of the manual control mode option by the user shown in FIG.10.

FIG. 13 illustrates a drawing showing a user interface screen with thesame control options as shown in FIG. 12 but with the user's fingerbeing additionally shown tapping/pressing the ISO control icon on thedisplay screen to select the ISO control option.

FIG. 14 illustrates a user interface screen displayed subsequent to theselection of the ISO control option in FIG. 13.

FIG. 15 illustrates a drawing showing a user interface screen with thesame control options as shown in FIG. 12 but with the user's fingerbeing additionally shown tapping/pressing the shutter speed control iconon the display screen to select shutter speed control option.

FIG. 16 illustrates a user interface screen displayed subsequent to theselection of the shutter speed control option in FIG. 15.

FIG. 17 illustrates a drawing showing a user interface screen with thesame control options as shown in FIG. 12 but with the user's fingerbeing additionally shown tapping/pressing the additional options icon onthe display screen to view additional available control options.

FIG. 18 illustrates a user interface screen displayed subsequent to userselection of the additional options icon shown in FIG. 17.

FIG. 19 illustrates a drawing showing user's selection of the exposurecontrol option on the user interface screen.

FIG. 20 illustrates a user interface screen displayed subsequent to theselection of the exposure control option shown in FIG. 19.

FIG. 21 illustrates a user interface screen displayed subsequent to theselection of the manual control mode option by the user shown in FIG. 10as an alternative to the user interface screen of FIG. 12.

FIG. 22 illustrates a user interface screen displayed subsequent to theselection of the ISO control option in FIG. 13.

FIG. 23 illustrates a subsequent user interface screen displayedsubsequent to the user's finger swipe in the downward direction on thedisplay screen of FIG. 22.

FIG. 24 illustrates a user interface screen displayed subsequent to theselection of the ISO control option in FIG. 13 wherein the user swipeshis/her finger in the upward direction as indicated by the arrow inorder to change the currently selected ISO value of “640”.

FIG. 25 illustrates a user interface screen displayed subsequent to theuser's finger swipe in the upward direction on the display screen ofFIG. 24.

FIG. 26 illustrates a user interface screen displayed when a useremploys a spread gesture to enlarge, e.g., zoom in, a preview imagedisplayed in the background.

FIG. 27 illustrates a user interface screen displayed subsequent to thescreen when the user continues to perform zoom in operation, e.g., byfinger spread action.

FIG. 28 illustrates a user interface screen displayed subsequent to thescreen when the user continues to perform zoom in operation, e.g.,continuing the finger spread action.

FIG. 29 illustrates a user interface screen 2900 displayed subsequent tothe screen 2800 in the case where the user still continues to performzoom in operation, e.g., continuing the finger spread action.

FIG. 30A is a first part of a flowchart illustrating the steps of anexemplary method of controlling a camera device, e.g., a camera deviceshown in any of the FIGS. 1-5, in accordance with an exemplaryembodiment.

FIG. 30B is a second part of the flowchart illustrating the steps of theexemplary method of controlling the camera device in accordance with anexemplary embodiment.

FIG. 30C is a third part of the flowchart illustrating the steps of theexemplary method of controlling the camera device performed along onespecific path of the method.

FIG. 30 comprises the combination of FIGS. 30A, 30B and 30C.

FIG. 31 illustrates a drawing showing an object, e.g., a cube with oneletter on each side, placed on a table and an exemplary camera devicewhich may be used to capture an image of the object.

FIG. 32 illustrates a drawing showing the camera and a displayed previewimage on the camera display screen at a time when the user has activatedthe exemplary orientation indicator tool, e.g., by selecting the OIoption icon.

FIG. 33 illustrates a drawing showing the object on the table and theexemplary camera device held at an angle with respect to the horizontalsurface of the table which is parallel to the ground.

FIG. 34 illustrates a drawing showing the camera and a displayed previewimage on the camera display screen corresponding to the capture scenarioillustrated in FIG. 33 and with the orientation indicator tool beingactivated.

FIG. 35 illustrates a drawing showing the object on the table and theexemplary camera device held directly facing the top face of the object.

FIG. 36 illustrates a drawing showing the camera and a displayed previewimage on the camera display screen corresponding to the capture scenarioillustrated in FIG. 35 and with the orientation indicator tool beingactivated.

FIG. 37 illustrates a drawing showing the object on the table and theexemplary camera device held such that the camera is both tilted, e.g.,with respect to the ground or table surface, and rotated, e.g.,anticlockwise with respect to a vertical axis extending from the topface of the object to the bottom face of the object or the table.

FIG. 38 illustrates a drawing showing the camera and a displayed previewimage on the camera display screen corresponding to the capture scenarioillustrated in FIG. 37 and with the orientation indicator tool beingactivated.

FIG. 39 illustrates a drawing showing the object on the table and theexemplary camera device held such that the camera is both tilted, e.g.,with respect to the ground or table surface, and rotated, e.g.,clockwise with respect to a vertical axis extending from the top face ofthe object to the bottom face of the object or the table.

FIG. 40 illustrates a drawing showing the camera and a displayed previewimage on the camera display screen corresponding to the capture scenarioillustrated in FIG. 39 and with the orientation indicator tool beingactivated.

FIG. 41A is a first part of a flowchart of an exemplary method inaccordance with an exemplary embodiment.

FIG. 41B is a second part of a flowchart of an exemplary method inaccordance with an exemplary embodiment.

FIG. 41C is a third part of a flowchart of an exemplary method inaccordance with an exemplary embodiment.

FIG. 41 comprises the combination of FIG. 41A, FIG. 41B and FIG. 41C.

FIG. 42 is a drawing of an exemplary touch screen display view of acamera device, including multiple optical chains, which may be, andsometimes is, displayed to a user of the camera device, e.g., to presentthe user with user control options, in accordance with an exemplaryembodiment.

FIG. 43 is a drawing an exemplary touch screen display view of a cameradevice, including multiple optical chains, which may be, and sometimesis, presented to a user of the camera device, e.g., to allow a userinput a depth of field setting, in accordance with an exemplaryembodiment.

FIG. 44 is a drawing an exemplary touch screen display view of a cameradevice, including multiple optical chains, which may be, and sometimesis, presented to a user of the camera device, e.g., to allow a user toinput a depth of field setting, in accordance with an exemplaryembodiment.

FIG. 45 is a drawing an exemplary touch screen display view of a cameradevice, including multiple optical chains, which may be, and sometimesis, presented to a user of the camera device, e.g., to allow a user toinput a depth of field setting, in accordance with an exemplaryembodiment.

FIG. 46 is a drawing an exemplary touch screen display view of a cameradevice, including multiple optical chains, which may be, and sometimesis, presented to a user of the camera device, e.g., to allow a user toselect an aspect ratio setting, in accordance with an exemplaryembodiment.

FIG. 47 is a drawing of an exemplary generated stored file in accordancewith an exemplary embodiment.

FIG. 48 is a drawing of an exemplary generated stored file includinguser selected information, e.g., user selected image control optionssettings and camera settings, and captured images in accordance with anexemplary embodiment.

FIG. 49A is a drawing of a first part of an assembly of modules whichmay be included in a camera device including multiple optical chains inaccordance with an exemplary embodiment.

FIG. 49B is a drawing of a second part of an assembly of modules whichmay be included in a camera device including multiple optical chains inaccordance with an exemplary embodiment.

FIG. 49 comprises the combination of FIG. 49A and FIG. 49B.

FIG. 50 illustrates an exemplary system including a camera device and animage processing system in accordance with an exemplary embodiment.

DETAILED DESCRIPTION

FIG. 1 illustrates an exemplary camera device 100 such as a digitalcamera, notepad with camera functionality, or cell phone with camerafunctionality, implemented in accordance with one exemplary embodimentof the present invention. The camera device 100, in some embodiments, isa portable device. In other embodiments, the camera device 100 is afixed device such as a wall mounted camera.

FIG. 1 illustrates the camera device 100 in block diagram form showingthe connections between various elements of the apparatus 100. Theexemplary camera device 100 includes a display device 102, a lightemitter module 104, an input device 106, an input state detection module148, an exposure and readout controller 150, e.g., a rolling shuttercontroller 150, a light control device 152, memory 108, a processor 110,a hardware assembly of modules 180, a wireless and/or wired interface114, e.g., a cellular interface, a Wi-Fi interface, and/or a USBinterface, an I/O interface 112, an accelerometer module 122, 3 axisgyro 192, and a bus 116 which are mounted in a housing represented bythe rectangular box touched by the line leading to reference number 100.The light emitter module 104 includes light emitting elements whichmaybe LEDs (Light Emitting Diodes) or other types of light emittingelements which can be individually controlled so that all the lightemitting elements need not be on at the same time. The input device 106may be, and in some embodiments is, e.g., keypad, touch screen, orsimilar device that may be used for inputting information, data and/orinstructions. The accelerometer module 122 includes accelerometer 1 124,accelerometer 2, 126 and accelerometer 3 128 which are arrayed onperpendicular axis providing a 3 axis accelerometer module. Thus, theaccelerometer module 122 can measure along 3 independent axis.

Similarly, the 3-axis gyro 192, which includes 194, 196 and 198 canmeasure rotation along each of 3 different axis. The output of theaccelerometer module 122 and the gyro module 192 can, and in someembodiments is, monitored with changes in accelerometer and gyro outputbeing interpreted and checked over time by processor 110 and/or zoomcontrol module, e.g., zoom controller 140, to detect changes inacceleration indicating motion in one or more directions. In someembodiments the input device 106 includes at least one zoom controlbutton that can be used to enable or disable camera zoom functionality.In some such embodiments when the zoom control button is in a depressedstate the camera zoom function is enabled while when the button is in aun-depressed state the camera zoom function is disabled. The input statedetection module 148 is configured to detect the state of the inputdevice, e.g., the zoom control button, to detect whether the button isin a depressed state or undepressed state. In some embodiments there isa status register in the camera device 100 that includes a bitindicating the state of the zoom control button detected by the statedetection module 148, e.g., whether it is in the depressed stateindicating that zoom is enabled or whether it is undepressed indicatingthat zoom is disabled.

The display device 102 may be, and in some embodiments is, a touchscreen, used to display images, video, information regarding theconfiguration of the camera device, and/or status of data processingbeing performed on the camera device. In the case where the displaydevice 102 is a touch screen, the display device 102 serves as anadditional input device and/or as an alternative to the separate inputdevice, e.g., buttons, 106. As will be discussed in some embodimentszooming operation can be controlled by pressing a zoom control sensor,e.g., a touch sensor. In some embodiments when the camera user touchesthe zoom control sensor the zoom functionality is enabled. For example afinger on the touch sensor activates/enables the zoom functionality. TheI/O interface 112 couples the display 102 and input device 106 to thebus 116 and interfaces between the display 102, input device 106 and theother elements of the camera which can communicate and interact via thebus 116.

In addition to being coupled to the I/O interface 112, the bus 116 iscoupled to the memory 108, processor 110, an optional autofocuscontroller 132, the wireless and/or wired interface 114, a zoom controlmodule 140, and a plurality of optical chains 130, e.g., X opticalchains also referred to herein as camera modules. In some embodiments Xis an integer greater than 2, e.g., 3, 4, 7 or a larger value dependingon the particular embodiment. The plurality of camera modules 130 may beimplemented using any of various camera module sets and/or arrangements.Images captured by individual optical chains in the plurality of opticalchains 130 can, and in various embodiments are, stored in memory 108,e.g., as part of the data/information 120 and processed by the processor110, e.g., to generate one or more composite images.

The X camera modules 131 through 133 may, and in various embodiments do,include camera modules having different focal lengths. Multiple cameramodules may be provided at a given focal length. For example, multiplecamera modules having a 35 mm equivalent focal length to a full frameDSLR camera, multiple camera modules having a 70 mm equivalent focallength to a full frame DSLR camera and multiple camera modules having a140 mm or 150 mm equivalent focal length to a full frame DSLR camera areincluded in an individual camera device in some embodiments. The variousfocal lengths are exemplary and a wide variety of camera modules withdifferent focal lengths may be used. Thus, in some embodiments thecamera modules with the largest focal lengths have a 150 mm focal lengthwhere the 150 mm focal length is a 35 mm film equivalent focal length.Accordingly, while 140 mm camera modules are mentioned in some locationsin this application in some embodiments 150 mm modules are used insteadof 140 mm focal length modules. The camera device 100 is to beconsidered exemplary. To the extent that other references are made to acamera or camera device with regard to some of the other figures, it isto be understood that at least in some embodiments the camera device orcamera will include the elements shown in FIG. 1 even if the elementsare not shown in a particular figure or embodiment. While in someembodiments all of the elements shown in FIG. 1 are included in thecamera device or camera, in other embodiments a subset of the elementsshown in FIG. 1 are included and the illustration of the elements inFIG. 1 is not intended to imply that a particular element is essentialor necessary in all embodiments.

As will be discussed below images from different camera modules capturedat the same time or during a given time period can be combined togenerate a composite image, e.g., an image having better resolution,frequency content and/or light range than an individual image capturedby a single one of the camera modules 131, 133.

Multiple captured images and/or composite images may, and in someembodiments are, processed to form video, e.g., a series of imagescorresponding to a period of time. The interface 114 couples theinternal components of the camera device 100 to an external network,e.g., the Internet, and/or one or more other devices e.g., memory orstand alone computer. Via interface 114 the camera device 100 can anddoes output data, e.g., captured images, generated composite images,and/or generated video. The output may be to a network or to anotherexternal device for processing, storage and/or to be shared. Thecaptured image data, generated composite images and/or video can beprovided as input data to another device for further processing and/orsent for storage, e.g., in external memory, an external device or in anetwork.

The interface 114 of the camera device 100 may be, and in some instancesis, coupled to a computer so that image data may be processed on theexternal computer. In some embodiments the external computer has ahigher computational processing capability than the camera device 100which allows for more computationally complex image processing of theimage data outputted to occur on the external computer. The interface114 also allows data, information and instructions to be supplied to thecamera device 100 from one or more networks and/or other externaldevices such as a computer or memory for storage and/or processing onthe camera device 100. For example, background images may be supplied tothe camera device to be combined by the camera processor 110 with one ormore images captured by the camera device 100. Instructions and/or dataupdates can be loaded onto the camera via interface 114 and stored inmemory 108.

The lighting module 104 in some embodiments includes a plurality oflight emitting elements, e.g., LEDs, which can be illuminated in acontrolled manner to serve as the camera flash with the LEDs beingcontrolled in groups or individually, e.g., in a synchronized mannerbased on operation of the rolling shutter and/or the exposure time. Forpurposes of discussion module 104 will be referred to as an LED modulesince in the exemplary embodiment LEDs are used as the light emittingdevices but as discussed above the invention is not limited to LEDembodiments and other light emitting sources may be used as well. Insome embodiments the LED module 104 includes an array of light emittingelements, e.g., LEDs. In some embodiments the light emitting elements inthe LED module 104 are arranged such that each individual LED and/or agroup of LEDs can be illuminated in a synchronized manner with rollingshutter operation. Light emitting elements are illuminated, in some butnot all embodiments, sequentially, so that different portions of an areaare illuminated at different times so that the full area need not beconsistently lighted during image capture. While all lighting elementsare not kept on for the full duration of an image capture operationinvolving the reading out of the full set of pixel elements of a sensor,the portion of area which is having its image captured, e.g., the scanarea, at a given time as a result of the use of a rolling shutter willbe illuminated thanks to synchronization of the lighting of lightemitting elements with rolling shutter operation. Thus, various lightemitting elements are controlled to illuminate at different times insome embodiments based on the exposure time and which portion of asensor will be used to capture a portion of an image at a given time. Insome embodiments the light emitting elements in the LED module 104include a plurality of sets of light emitting elements, each set oflight emitting elements corresponding to a different image area which itilluminates and which is captured by a different portion of the imagesensor. Lenses may, and in some embodiments are used to direct the lightfrom different light emitting elements to different scene areas whichwill be captured by the camera through the use of one or more cameramodules.

The camera device 100 also includes a user interface module 179 whichmaybe and sometimes is implemented in hardware, e.g., as a circuit suchas an ASIC, while in other embodiments the user interface 179 isimplemented in software which, when executed by the processor 110 causesthe processor 110 to control the camera device to implement one or moreof the user interface control methods and features described herein.

The rolling shutter controller 150 is an electronic shutter thatcontrols reading out of different portions of one or more image sensorsat different times. Each image sensor is read one row of pixel values ata time and the various rows are read in order. As will be discussedbelow, the reading out of images captured by different sensors iscontrolled in some embodiments so that the sensors capture a scene areaof interest, also sometimes referred to as an image area of interest, ina synchronized manner with multiple sensors capturing the same imagearea at the same time in some embodiments.

While an electronic rolling shutter is used in most of the embodiments,a mechanical rolling shutter may be used in some embodiments.

The light control device 152 is configured to control light emittingelements (e.g., included in the LED module 104) in a synchronized mannerwith the operation of the rolling shutter controller 150. In someembodiments the light control device 152 is configured to controldifferent sets of light emitting elements in the array to emit light atdifferent times in a manner that is synchronized with the timing of therolling shutter 150. In some embodiments the light control device 152 isconfigured to control a first set of light emitting elementscorresponding to a first image area to output light during a first timeperiod, the first time period being determined based on the timing ofthe rolling shutter and being a period of time during which a firstportion of the sensor is exposed for image capture. In some embodimentsthe light control device 152 is further configured to control a secondset of light emitting elements corresponding to a second image area tooutput light during a second time period, the second time period beingdetermined based on the timing of the rolling shutter and being a periodof time during which a second portion of the sensor is exposed for imagecapture. In some embodiments the first time period includes at least aportion of time which does not overlap the second time period.

In some embodiments the light control device 152 is further configuredto control an Nth set of light emitting elements corresponding to an Nthimage area to output light during a third time period, said Nth timeperiod being determined based on the timing of the rolling shutter andbeing a period of time during which an Nth portion of the sensor isexposed for image capture, N being an integer value corresponding to thetotal number of time periods used by said rolling shutter to completeone full read out of total image area.

In some embodiments the light control device 152 is further configuredto control the second set of light emitting elements to be off duringsaid portion of time included in the first period of time which does notoverlap said second period of time. In some embodiments the lightcontrol device is configured to determine when the first set and saidsecond set of light emitting elements are to be on based on an exposuresetting. In some embodiments the light control device is configured todetermine when said first set and said second set of light emittingelements are to be on based on an amount of time between read outs ofdifferent portions of said sensor. In some embodiments the differentsets of light emitting elements in the plurality of light emittingelements are covered with different lenses. In some such embodiments thelight control device 152 is further configured to determine which setsof light emitting elements to use based on an effective focal lengthsetting being used by the camera device.

The accelerometer module 122 includes a plurality of accelerometersincluding accelerometer 1 124, accelerometer 2 126, and accelerometer 3128. Each of the accelerometers is configured to detect cameraacceleration in a given direction. Although three accelerometers 124,126 and 128 are shown included in the accelerometer module 122 it shouldbe appreciated that in some embodiments more than three accelerometerscan be used. Similarly the gyro module 192 includes 3 gyros, 194, 196and 198, one for each axis which is well suited for use in the 3dimensional real world environments in which camera devices are normallyused. The camera acceleration detected by an accelerometer in a givendirection is monitored. Acceleration and/or changes in acceleration, androtation indicative of camera motion, are monitored and processed todetect one or more directions, of motion e.g., forward camera motion,backward camera motion, etc. As discussed below, theacceleration/rotation indicative of camera motion can be used to controlzoom operations and/or be provided in some cases to a camera mount whichcan then take actions such as rotating a camera mount or rotating acamera support to help stabilize the camera.

The camera device 100 may include, and in some embodiments does include,an autofocus controller 132 and/or autofocus drive assembly 134. Theautofocus drive assembly 134 is, in some embodiments, implemented as alens drive. The autofocus controller 132 is present in at least someautofocus embodiments but would be omitted in fixed focus embodiments.The autofocus controller 132 controls adjustment of at least one lensposition in one or more optical chains used to achieve a desired, e.g.,user indicated, focus. In the case where individual drive assemblies areincluded in each optical chain, the autofocus controller 132 may drivethe autofocus drive of various optical chains to focus on the sametarget.

The zoom control module 140 is configured to perform a zoom operation inresponse to user input. The processor 110 controls operation of thecamera device 100 to control the elements of the camera device 100 toimplement the steps of the methods described herein. The processor maybe a dedicated processor that is preconfigured to implement the methodsof the present invention. However, in many embodiments the processor 110operates under direction of software modules and/or routines stored inthe memory 108 which include instructions that, when executed, cause theprocessor to control the camera device 100 to implement one, more or allof the methods described herein. Memory 108 includes an assembly ofmodules 118 wherein one or more modules include one or more softwareroutines, e.g., machine executable instructions, for implementing theimage capture, image generation and/or image data processing methods ofthe present invention. Individual steps and/or lines of code in themodules of 118 when executed by the processor 110 control the processor110 to perform steps of the method of the invention, e.g., generatingdepth map, determining maximum expected frequencies and/or filteringimage portions, in accordance with the invention. When executed byprocessor 110, the assembly of modules 118 cause at least some data tobe processed by the processor 110 in accordance with the method of thepresent invention, e.g., filtering image portions in accordance with theinvention. The assembly of modules 118 includes a mode control modulewhich determines, e.g., based on user input which of a plurality ofcamera device modes of operation are to be implemented. In differentmodes of operation, different camera modules 131, 133 may and often arecontrolled differently based on the selected mode of operation. Forexample, depending on the mode of operation different camera modules mayuse different exposure times. Alternatively, the scene area to which thecamera module is directed and thus what portion of a scene is capturedby an individual camera module may be changed depending on how theimages captured by different camera modules are to be used, e.g.,combined to form a composite image and what portions of a larger sceneindividual camera modules are to capture during the user selected orautomatically selected mode of operation. In some embodiments, theoperations performed by the processor when executing the instructionsfrom one or more assembly of modules is instead performed by a hardwaremodule which performs the same functionality and is included in thehardware assembly of modules 180.

The resulting data and information (e.g., captured images of a scene,combined or composite images of a scene, filtered images etc.) arestored in data/information block 120 for future use, additionalprocessing, and/or output, e.g., to display device 102 for display or toanother device for transmission, processing and/or display. In someembodiments the data/information block 120 further includes opticalchain information, e.g., optical characteristics, corresponding to theplurality of optical chains 130 in the device 100. If one or moreparameters/settings in the optical characteristics of a camera modulechanges then the corresponding optical chain information stored in thedata/information 120 is updated. The memory 108 includes different typesof memory for example, Random Access Memory (RAM) in which the assemblyof modules 118 and data/information 120 may be, and in some embodimentsare stored for future use. Read only Memory (ROM) in which the assemblyof modules 118 may be stored for power failures. Non-volatile memorysuch as flash memory for storage of data, information and instructionsmay also be used to implement memory 108. Memory cards may be added tothe device to provide additional memory for storing data (e.g., imagesand video) and/or instructions such as programming. Accordingly, memory108 may be implemented using any of a wide variety of non-transitorycomputer or machine readable mediums which serve as storage devices.

Having described the general components of the camera device 100 withreference to FIG. 1, various features relating to the plurality ofoptical chains 130 will now be discussed with reference to FIGS. 2 and 3which show the camera device 100 from front and side perspectives,respectively. Dashed line 101 of FIG. 2 indicates a cross section line.

Box 117 represents a key and indicates that OC=optical chain, e.g.,camera module, and each L1 represents an outermost lens in an opticalchain. Box 119 represents a key and indicates that S=sensor, F=filter,L=lens, L1 represents an outermost lens in an optical chain, and L2represents an inner lens in an optical chain. While FIG. 3 shows onepossible implementation of optical chains, as will be discussed below,other embodiments are possible and the optical chains may include one ormore light redirection elements in addition to the elements shown inFIG. 3. The lenses of different optical chains may have differentshapes, e.g., with round apertures being used for some lenses andnon-round apertures being used for other lenses. However, in someembodiments lenses with round apertures are used for each of the opticalchains of a camera device.

FIG. 2 shows the front of the exemplary camera device 100. Rays of light131, which is light toward the front of the camera assembly, shown inFIG. 1 may enter the lenses located in the front of the camera housing.From the front of camera device 100, the camera device 100 appears as arelatively flat device with the outer rectangle representing the camerahousing and the square towards the center of the camera representing theportion of the front camera body in which the plurality of opticalchains 130 is mounted. Note that while outer opening shown in FIG. 2 areshown as having circular apertures which are the same size, as will bediscussed below different size openings may be used for differentoptical chains, e.g., depending on the focal length with optical chainshaving larger focal lengths normally including outer openings withlarger apertures than optical chains with small focal lengths.

FIG. 3, which shows a side perspective of camera device 100, illustratesthree of the seven optical chains (OC 1 121, OC 7 145, OC 4 133) of theset of optical chains 130, display 102 and processor 110. OC 1 121includes an outer opening 103, a light redirection element 252, e.g., amirror, an inner lens L2 125, a filter 123 and a sensor 127. In someembodiments the OC 1 121 further includes lens drive (LD) 129 forcontrolling the position of lens L2 125 for zooming and/or auto focusoperation purposes and a mirror drive (MD) 129′ for controlling thepositioning of the light reflection element 252 as desired to deflectlight. The outer opening 103 serves as an aperture of the camera moduleOC 121, e.g., for entry of light into OC 121. The exposure and read outcontroller 150 is not shown in the figure but is used for controllingthe read out of rows of pixel values form the sensors' 127, 151 and 139in a synchronized manner, e.g., taking into consideration the scene areabeing captured by the individual sensors. The LD 129 includes a motor orother drive mechanism which can move the lens, barrel or cylinderhousing one or more lenses, or sensor, to which it is connected therebyallowing for an alteration to the light path by moving one or moreelements relative to the other elements of the optical chain to whichthe LD is coupled. While the LD 129 is shown coupled, e.g., connected,to the lens L2 125 and thus can move the position of the lens L2, e.g.,as part of a zooming or autofocus operation, in other embodiments the LD129 is coupled to a cylindrical or barrel shape component which is partof the optical chain or to the sensor 127. Thus, the lens drive 129 canalter the relative position of a lens to the sensor 127, e.g., to changethe distance between the sensor 127 and the lens 125 as part of azooming and/or focus operation. The MD includes a motor or other drivemechanism which can control the relative angle of reflection element 252allowing for alteration of angle of redirection of incident light.

OC 7 145 includes an outer opening 115, a light redirection element 231,an inner lens L2 149, a filter 147, and a sensor 151. OC 7 145 furtherincludes LD 153 for controlling the position of lens L2 149 and a and amirror drive (MD) 153′ for controlling the positioning of the lightreflection element 231. The LD 153 includes a motor or other drivemechanism which can move the lens, barrel, cylinder, sensor or otheroptical chain element to which it is connected.

OC 4 133 includes an outer opening 109, a light redirection element 235,an inner lens L2 137, a filter 135 and a sensor 139. OC 4 133 includesLD 141 for controlling the position of lens L2 137 and MD 141′ forcontrolling the positioning of the light reflection element 235. The LD153, 141 and MD 153′, 141′ include a motor or other drive mechanism andoperates in the same or similar manner as the other drives of the otheroptical chains discussed above. In some embodiments each of the filters123, 147 and 135 is an infrared (IR) filter. While only three of the OCsare shown in FIG. 3 it should be appreciated that the other OCs of thecamera device 100 may, and in some embodiments do, have the same orsimilar structure and/or may include other elements such as lightredirection devices. Thus, differences between the multiple opticalchains of the camera device 100 are possible and, in some embodiments,are present to allow for a variety of focal lengths to be supported in asingle camera device through the use of multiple optical chains whichcan be operated in parallel.

FIG. 3 and the optical chains (OCs), also sometimes referred to ascamera modules, illustrated therein are illustrative of the generalstructure of OCs used in various embodiments. However, numerousmodifications and particular configurations are possible. Whilereference to elements of FIG. 3 may be made, it is to be understood thatthe OCs (camera modules) in a particular embodiment will be configuredas described with regard to the particular embodiment and that variousdifferent camera modules are often used in single camera device. FIG. 5shows optical chains, e.g., camera modules, which include lightredirection devices. Such modules can be used alone or in combinationwith other modules such as the ones shown in FIGS. 3 and 4 or otherfigures of the present application.

While a filter may be of a particular color or used in some opticalchains, filters need not be used in all optical chains and may not beused in some embodiments. In embodiments where the filter is expresslyomitted and/or described as being omitted or an element which allows alllight to pass, while reference may be made to the OCs of FIG. 3 itshould be appreciated that the filter will be omitted in an embodimentwhere it is indicated to be omitted or of such a nature that it allows abroad spectrum of light to pass if the embodiment is indicated to have abroadband filter. In some embodiments one or more light redirectionelements, e.g., mirrors, such as elements 252, 231, 235 shown in FIG. 3,are included in OCs for light to be redirected, e.g., to increase thelength of the optical path or make for a more convenient internalcomponent configuration. It should be appreciated that each of the OCs121, 145, 133, shown in FIG. 3 will have their own optical axis. In theexample, each optical axis passes through the outer openings 103, 115,or 109 at the front of the optical chain and passes through the OC tothe corresponding sensor 127, 151, 139.

While the processor 110 is not shown being coupled to the LD, andsensors 127, 151, 139 it is to be appreciated that such connectionsexist and are omitted from FIG. 3 to facilitate the illustration of theconfiguration of the exemplary OCs.

As should be appreciated the number and arrangement of lens, filtersand/or mirrors can vary depending on the particular embodiment and thearrangement shown in FIG. 3 is intended to be exemplary and tofacilitate an understanding of various features rather than to belimiting in nature.

The front of the plurality of optical chains 130 is visible in FIG. 2with the outermost opening of each optical chain appearing as a circlerepresented using a solid line (OC 1 opening 103, OC 2 opening 105, OC 3opening 107, OC 4 opening 109, OC 5 opening 111, OC 6 opening 113, OC 7opening 115). In the FIG. 2 example, the plurality of optical chains 130include seven optical chains, OC 1 121, OC 2 157, OC 3 159, OC 4 133, OC5 171, OC 6 173, OC 7 145, which include openings 103, 105, 107, 109,111, 113, 115), respectively, represented by the solid circles shown inFIG. 2. While the outer opening may be a circular opening in someembodiments, in some other embodiments the entry point for the lightinto the optical chains has a plastic element covering the opening. Theouter openings of the optical chains are arranged to form a patternwhich is generally circular in the FIG. 2 example when viewed as a unitfrom the front. While a circular arrangement is used in someembodiments, non-circular arrangements are used and preferred in otherembodiments. In some embodiments while the overall pattern is generallyor roughly circular, different distances to the center of the generalcircle and/or different distances from one lens to another isintentionally used to facilitate generation of a depth map and blockprocessing of images which may include periodic structures such asrepeating patterns without the need to identify edges of the repeatingpattern. Such repeating patterns may be found in a grill or a screen.

The overall total light capture area corresponding to the multiplelenses of the plurality of optical chains OC 1 to OC 7, also sometimesreferred to as optical camera modules, can, in combination, approximatethat of a lens having a much larger opening but without requiring asingle lens having the thickness which would normally be necessitated bythe curvature of a single lens occupying the area which the lensesoccupy.

While seven optical chains are shown in FIG. 2, it should be appreciatedthat other numbers of optical chains are possible. For example, in someembodiments, seventeen camera modules are used in a single camera devicein some embodiments. Camera devices including even larger numbers ofoptical chains are also possible.

The use of multiple optical chains has several advantages over the useof a single optical chain. Using multiple optical chains allows fornoise averaging. For example, given the small sensor size there is arandom probability that one optical chain may detect a different number,e.g., one or more, photons than another optical chain. This mayrepresent noise as opposed to actual human perceivable variations in theimage being sensed. By averaging the sensed pixel values correspondingto a portion of an image, sensed by different optical chains, the randomnoise may be averaged resulting in a more accurate and pleasingrepresentation of an image or scene than if the output of a singleoptical chain was used.

Given the small size of the optical sensors (e.g., individual pixelelements) the dynamic range, in terms of light sensitivity, is normallylimited with the sensors becoming easily saturated under brightconditions. By using multiple optical chains corresponding to differentexposure times the dark portions of a scene area can be sensed by thesensor corresponding to the longer exposure time while the lightportions of a scene area can be sensed by the optical chain with theshorter exposure time without getting saturated. Pixel sensors of theoptical chains that become saturated as indicated by a pixel valueindicative of sensor saturation can be ignored, and the pixel value fromthe other, e.g., less exposed, optical chain can be used withoutcontribution from the saturated pixel sensor of the other optical chain.Weighting and combining of non-saturated pixel values as a function ofexposure time is used in some embodiments. By combining the output ofsensors with different exposure times a greater dynamic range can becovered than would be possible using a single sensor and exposure time.

FIG. 3 is a cross section perspective of the camera device 100 shown inFIGS. 1 and 2. Dashed line 101 in FIG. 2 shows the location within thecamera device to which the cross section of FIG. 3 corresponds. From theside cross section, the components of the first, seventh and fourthoptical chains are visible.

As illustrated in FIG. 3 despite including multiple optical chains thecamera device 100 can be implemented as a relatively thin device, e.g.,a device less than 2, 3 or 4 centimeters in thickness in at least someembodiments. Thicker devices are also possible, for example devices withtelephoto lenses, and are within the scope of the invention, but thethinner versions are particularly well suited for cell phones and/ortablet implementations. As will be discussed below, various techniquessuch as the use of light redirection elements and/or non-circular lensescan be used in conjunction with small sensors, such as those commonlyused in handheld cameras, to support relatively large focal lengths,e.g., camera modules of 150 mm equivalent focal length to a full frameDSLR camera, 300 mm equivalent focal length to a full frame DSLR cameraor above in a relatively thin camera device format.

As illustrated in the FIG. 3 diagram, the display device 102 may beplaced behind the plurality of optical chains 130 with the processor110, memory and other components being positioned, at least in someembodiments, above or below the display and/or optical chains 130. Asshown in FIG. 3, each of the optical chains OC 1 121, OC 7 145, OC 4 133may, and in some embodiments do, include an outer opening, a lightredirection element such as a mirror or prism, a filter F, and a lens L2which proceed a sensor S which captures and measures the intensity oflight which passes through the outer opening serving as the aperture,the lens L2 and the filter F to reach the sensor S. The filter may be acolor filter or one of a variety of other types of light filters or maybe omitted depending on the particular optical chain embodiment orconfiguration. In some embodiments the filter is an IR filter.

Note that while supporting a relatively large light capture area andoffering a large amount of flexibility in terms of color filtering andexposure time, the camera device 100 shown in FIG. 3 is relatively thinwith a thickness that is much less, e.g., ⅕th, 1/10th, 1/20th or evenless than the overall side to side length or even top to bottom lengthof the camera device visible in FIG. 2.

FIG. 4 illustrates a camera device 200 implemented in accordance withthe invention. The FIG. 4 camera device 200 includes many or all of thesame elements shown in the device 100 of FIGS. 1-3. Exemplary cameradevice 200 includes a plurality of optical chains (OC 1 205, OC 2 207, .. . , OC X 209, a processor 211, memory 213 and a display 215, coupledtogether. OC 1 205 includes outer opening 251, a light redirectionelement R 252, a hinge (or mirror) drive MD 291, an inner lens L2 253, afilter 255, sensor 1 257, and LD 259. The MD 291 can be used to move aposition of a hinge to which the light redirection device (R) 252, e.g.,mirror, is mounted and thus move the mirror to change the scene area towhich the module 205 is directed without moving the optical chain 205.Moving (e.g., rotating about a hinge) the mirror 252 to change the scenearea to which the module 205 is directed is especially useful in anembodiment where the outer opening 251 is a plane piece of glass or aplastic piece with no optical power as is the case in some embodiments.

The optical chains shown in FIG. 4 can be arranged in various positionswithin the camera 200. The elements in FIG. 5 which are the same asthose shown in FIG. 4 are identified using the same references numbersand will not be described again. FIG. 5 shows the configuration of theoptical chains in an arrangement where light enters via the front orface of the camera 200 and is redirected to sensors 257, 269, 281, ofthe first through third camera modules respectively, mounted on theinside top portion of the camera housing which forms the outer portionof camera 200.

As can be seen in the FIG. 5 embodiment, light entering in thehorizontal dimension is redirected upward in the vertical. For example,light entering through outer opening 251 of the first optical chain 205is redirected upward by mirror 252 so that it passes though the innerlens 253 and the filter 255 as it travels towards sensor 257. An opticalchain such as the first optical chain 205, that has a light redirectionelement, such as the element 252, can be divided, for purposes ofdiscussion, into two parts, Part A and Part B. Part A consists of allthose elements in the optical chain that are in the light path beforethe light redirection element 252 and Part B consists of all the opticalelements (including the image sensor) that are in the light path afterthe light redirection element. The optical axis of the optical chain 205as seen from outside the camera is the optical axis 291 of Part A. Lighttraveling into the optical chain 205 along the optical axis 291 will beredirected upward along the optical axis 293 of Part B of the firstoptical chain.

In one particular exemplary embodiment of the optical chain 205, Part Acontains no optical elements with any optical power, e.g., Part Acontains plane glass or filters but no lenses. In this case the opticalaxis of the optical chain as seen from outside the camera is simplyalong a light path that gets redirected along the optical axis 293 ofPart B by the light redirection element. In some embodiments one or morelenses 253 are included in Part B of the optical chain which have anoptical power. Thus, it should be appreciated that in at least someembodiments the outer opening 251 may be implemented as a flat glassplate or relatively flat plastic or glass element which does notprotrude from the surface of the camera 200. This reduces the risk ofscratches and also reduces the possibly that an outer portion which iscovering or forming the opening will get caught when inserting orremoving it from a pocket or case as might be the case if the opening iscovered by a curved lens protruding from the camera.

It should be appreciated that the optical axis of the second and thirdcamera modules are similar to that of the first optical module 205 andthat the components of the optical chains may also be grouped into twoparts, Part A which corresponds to components proceeding the mirror ofthe optical chain and Part B which corresponds to components subsequentthe mirror of the optical chain. From the perspective of the opticalpath of an optical chain, the optical path like the components may begrouped as Part A and Part B with the mirror providing the transitionpoint between Part A of an optical path and Part B of the optical path.

In some but not all embodiments, processor 211 of camera device 200 ofFIG. 4 is the same as or similar to processor 110 of device 100 of FIG.1, memory 213 of device 200 of FIG. 4 is the same as or similar to thememory 108 of device 100 of FIG. 1, the zoom control module 214 ofdevice 200 is the same as or similar to the zoom control module 140 ofdevice 100, the accelerometer module 216 of device 200 is the same as orsimilar to the accelerometer module 122 of device 100 and display 215 ofdevice 200 of FIG. 4 is the same as or similar to the display 102 ofdevice 100 of FIG. 1.

OC 2 207 includes outer opening 263, light redirection device 231,mirror drive 293, inner lens 265, filter 267, sensor 2 269, and LD 271.OC N 209 includes outer opening 275, light redirection device 235,mirror drive 295, inner lens 277, filter 279, sensor N 281, and LD 283.The exposure and read out controller 150 controls sensors to read out,e.g., rows of pixel values, in a synchronized manner while alsocontrolling the exposure time. In some embodiments the exposure and readout controller 150 is a rolling shutter controller including an exposurecontroller 287 and a sensor read out controller 289. An autofocuscontroller 152 is included to control the lens drives 259, 271 and 283in some embodiments.

In the FIG. 4 embodiment the optical chains (optical chain 1 205,optical chain 2 207, . . . , optical chain N 209) are shown asindependent assemblies with the lens drive of each module being aseparate LD element (LD 259, LD 271, LD 283), respectively. Each of theLDs shown adjusts the position of the corresponding lens to which it isconnected as part of a zooming and/or focus operation. In someembodiments the LD controls the position of a lens and/or sensor inwhich case the LD is connected to both a lens support mechanism or lensand the sensor.

In FIG. 4, the structural relationship between the mirror and variouslenses and filters which precede the sensor in each optical chain can beseen more clearly than in some of the other figures. While fourelements, e.g. an opening, lens (see columns 201 and 203), a lightredirection device R (see col. 217), and the filter (corresponding tocolumn 202) are shown in FIG. 4 before each sensor, it should beappreciated that a much larger combinations (e.g., numbers) of lenses,light redirection elements and/or filters may precede the sensor of oneor more optical chains with anywhere from 2-10 elements being common andan even larger number of elements being used in some embodiments, e.g.,high end embodiments and/or embodiments supporting a large number offilter and/or lens options. Furthermore it should be appreciated thatall illustrated elements need not be included in all optical chains. Forexample, in some embodiments optical chains having relatively shortfocal lengths may be implemented without the use of a light redirectionelement being used, e.g., to redirect the light by 90 degrees, since theoptical chain with a short focal length can be implemented in a straightbut still relatively compact manner given the short focal length.

In some but not all embodiments, optical chains are mounted in thecamera device with some, e.g., the shorter focal length optical chainsextending in a straight manner from the front of the camera devicetowards the back. However, in the same camera, longer focal lengthcamera modules may and sometimes do include light redirection deviceswhich allow at least a portion of the optical path of a camera module toextend sideways allowing the length of the optical axis to be longerthan the camera is deep. The use of light redirection elements, e.g.,mirrors, is particularly advantageous for long focal length cameramodules given that the overall length of such modules tends to be longerthan that of camera modules having shorter focal lengths. A camera mayhave a wide variety of different camera modules some with lightredirection elements, e.g., mirrors, and others without mirrors. Filtersand/or lenses corresponding to different optical chains may, and in someembodiments are, arranged in planes, e.g. the apertures of the outermostlenses may be configured in a plane that extends parallel to the face ofthe camera, e.g., a plane in which the front of the camera both extendsvertically and horizontally when the camera is in a vertical directionwith the top of the camera both being up.

FIG. 6-29 illustrate various exemplary user interface screens that maybe displayed to a user of the camera device shown in FIG. 1 via whichthe user may control the camera operations in accordance with theinvention.

FIG. 6 shows an exemplary user interface screen 600 displayed to theuser upon switching the camera device on in a normal mode of operation,in accordance with one embodiment. In some embodiments the default modeof operation of the camera is the normal mode in which while a previewimage of environments/objects scanned by the camera are displayed butcontrol options for controlling camera settings are not displayed. Ascan be seen in FIG. 6, the user interface screen 600 simply displays apreview image (not illustrated graphically but indicated by the word“Preview Image”) while not displaying other options and/or controlfeatures.

FIG. 7 shows an exemplary user interface screen 700 with a user's finger706 tapping or being swiped on an area 702 of the display screen. Inaccordance with one aspect of some embodiments when the camera is innormal mode and the user touches, e.g., taps and/or swipes, the area 702of the display screen, the device switches to a control mode and one ormore control options are subsequently presented as will be discussed inmore detail later. While in FIG. 7 example the area 702 is shown as asmall portion on the top right corner of the display screen it should beunderstood that this is merely exemplary. The area 702 may be bigger,smaller or may correspond to a different portion of the display screen,e.g., bottom right, top left, bottom left etc. Furthermore the userinput for controlling switching from normal mode to control mode may beprovided in other predetermined manners too, for example, by swiping thefingers in a particular fashion, by tapping the screen a predeterminednumber of times and/or in a number of other manners and the camera isable to detect such user inputs.

FIG. 8 shows an exemplary user interface screen 800 displayed to theuser upon the camera detecting user input, e.g., finger tap, on thedisplay screen over the area 702 shown in FIG. 7. The user interfacescreen 800 shows the control options displayed to the user on the screensubsequent to the camera detecting user input indicating a desire toview control options and/or desire to switch to a control mode ofoperation. In some embodiments various icons corresponding to controloptions are displayed from top to bottom arranged vertically on theright side of the display screen 800. However a variety of otherdifferent arrangements are possible. The displayed control optionsinclude an option to select an auto control mode represented by autoicon 802, an option to select a manual control mode represented bymanual icon 804 and an option to select video mode represented by videoicon 806. The user may tap any of these icons to select thecorresponding option. It should be appreciated that while variousoptions are displayed the display screen continues to show the previewimage in the background.

FIG. 9 illustrates a drawing 900 showing a display screen with the samecontrol options as shown in FIG. 8 but with a user's finger 704 beingadditionally shown tapping/pressing the auto mode icon 802 on thedisplay screen.

FIG. 10 illustrates a user interface screen 1000 displayed subsequent tothe selection of the auto control mode option by the user shown in FIG.9. Subsequent to the selection of auto control mode option, an auto modeindicator 1002, an exposure control option 1004 and an additionaloptions bar 1006 is displayed on the screen as shown in FIG. 10. Theauto mode indicator 1002 is an indicator displayed to indicate/notifythe user that the camera is in auto control mode. In accordance with oneaspect of some embodiments in auto control mode various camera controlssuch as shutter speed, brightness, ISO, exposure etc., are automaticallycontrolled and limited options are available to the user formodification. For example while many other options are automaticallycontrolled in the auto control mode, still in some embodiments exposurecontrol option 1004 is provided to the user with limited selectableexposure settings for the user to select if desired. The additionaloptions bar 1006 is available to allow the user to view more availableoptions beyond the ones displayed on the user interface screen 1000 ofFIG. 10.

FIG. 11 illustrates a drawing 1100 showing a user interface screen 1100with the same control options as shown in FIG. 8 but with a user'sfinger 704 being additionally shown tapping/pressing the manual modeicon 804 on the display screen.

FIG. 12 illustrates a user interface screen 1200 displayed subsequent tothe selection of the manual control mode option by the user shown inFIG. 10. Subsequent to the selection of manual control mode option, amanual mode indicator 1202, an ISO control option 1204, a shutter speedcontrol option 1206 and an additional options bar 1208 is displayed onthe screen as shown in FIG. 12. The manual mode indicator 1002 is anindicator displayed to indicate/notify the user that the camera is inmanual control mode. In accordance with one aspect of some embodimentsin the manual control mode various camera controls are available to theuser and can be set/configured by the user as desired. Among theavailable control options in the manual mode is an ISO control optionrepresented by the ISO icon 1204, a shutter speed control optionrepresented by the icon 1206. The ISO control option 1204 allows theuser to control the sensitivity of the image sensor by increasing ordecreasing a selectable ISO value. The shutter speed control option 1206allows the user to control the shutter speed by increasing or decreasinga selectable shutter speed value. The additional options bar 1208 isavailable to allow the user to view more available options beyond theones displayed on the user interface screen 1200 of FIG. 12.

FIG. 13 illustrates a drawing showing a user interface screen 1300 withthe same control options as shown in FIG. 12 but with the user's finger704 being additionally shown tapping/pressing the ISO control icon 1204on the display screen to select the ISO control option.

FIG. 14 illustrates a user interface screen 1400 displayed subsequent tothe selection of the ISO control option in FIG. 13. Subsequent to theselection of the ISO control option a set of user selectable controlvalues, e.g., ISO control values in FIG. 14, are displayed to the useron the display screen 1400 as shown in FIG. 14. As illustrated in thefigure, in some embodiments the user selectable control values aredisplayed as if arranged on the surface of a rotating wheel with acurrently selected value being closest to the user, e.g., in the centerand displayed in larger font, and other selectable values above andbelow the currently selected value being shown using a smaller size asif further away from the user. In the illustrated example of FIG. 14 itcan be seen that the currently selected value 1406 (e.g., ISO value of640 in the example) that will appear to be closest to the user is in thecenter and appears more prominent than other displayed values. The usermay swipe his finger on the screen in upward or downward direction tochange the ISO value with a new currently selected ISO value moving tothe position in the center adjacent the current selection pointer 1404,e.g., as part of a simulated wheel rotation, and the previously selectedISO value moving away from the center position. In some embodiments avibration, also sometimes referred to as a haptic, is generated eachtime a new value moves to the center of the screen as part of thesimulated wheel rotation. An ISO indicator 1402 can also been seen onthe display screen indicating that control values being displayedcorrespond to ISO control settings. Furthermore the ISO control optionicon 1204 is also displayed on the right side. If the user taps the ISOcontrol option icon 1204 the display screen shown in FIG. 14 switchesback to the main control options display screen 1200 of FIG. 12.

FIG. 15 illustrates a drawing showing a user interface screen 1500 withthe same control options as shown in FIG. 12 but with the user's finger704 being additionally shown tapping/pressing the shutter speed controlicon 1206 on the display screen to select shutter speed control option.

FIG. 16 illustrates a user interface screen 1600 displayed subsequent tothe selection of the shutter speed control option in FIG. 15. Subsequentto the selection of the shutter speed control option a set of userselectable control values, e.g., shutter speed control values in FIG. 16example, are displayed to the user on the display screen 1600 as shownin FIG. 16. As illustrated in the figure, the user selectable shutterspeed control values are displayed as if arranged on the surface of arotating wheel with a currently selected value being closest to theuser, e.g., in the center and displayed in larger font, and otherselectable values above and below the currently selected value beingshown using a smaller size as if further away from the user. In theillustrated example of FIG. 16 it can be seen that the currentlyselected shutter speed value 1606 is 1/125. In some embodiment avibration (haptic) occurs each time a value moves to the center of thescreen as part of the simulated wheel rotation. The user may swipe hisfinger on the screen in upward or downward direction to change theshutter speed with a new currently selected shutter speed value movingto the position in the center adjacent the current selection pointer1604 and the previously selected shutter speed value moving away fromthe center position. A shutter speed indicator 1602 can also been seenon the display screen indicating that control values being displayedcorrespond to the shutter speed control settings. Furthermore theshutter speed control option icon 1206 is also displayed on the rightside portion of the display. In some embodiments if the user taps theshutter speed control option icon 1206 the display screen shown in FIG.16 switches back to the main control options display screen 1200 of FIG.12.

FIG. 17 illustrates a drawing showing a user interface screen 1700 withthe same control options as shown in FIG. 12 but with the user's finger704 being additionally shown tapping/pressing the additional optionsicon 1208 on the display screen to view additional available controloptions. Upon selecting, e.g., tapping/pressing, the additional optionsicon 1208 additional available control options, if any, are displayed.

FIG. 18 illustrates a user interface screen 1800 displayed subsequent touser selection of the additional options icon 1208 shown in FIG. 17.Subsequent to the selection of the additional options 1208 one or moreremaining available control options are displayed on the display screen1800 as shown in FIG. 16. As illustrated in the figure, in additional tothe manual mode indicator 1202 an exposure control icon 1804 indicatingexposure control option and a OI (orientation indicator) icon 1806 toopen an orientation indicator tool, are displayed. The exposure controloption can be explored by selecting the exposure control icon 1804 andallows the user to select and/or modify exposure settings of the camera.The orientation indicator tool can be explored by selecting the OI icon1806 and provides the user with feedback about the orientation of thecamera device.

FIG. 19 illustrates a drawing showing user's selection of the exposurecontrol option on the user interface screen 1900. The drawing shows theuser's finger 704 tapping/pressing the exposure control icon 1804 on thedisplay screen 1900 to view user selectable exposure control values.

FIG. 20 illustrates a user interface screen 2000 displayed subsequent tothe selection of the exposure control option 1804 shown in FIG. 19.Subsequent to the selection of the exposure control option a set of userselectable control values, e.g., exposure control values in FIG. 20example, are displayed to the user on the display screen 2000 as shown.Similar to what has been discussed with regard to FIGS. 14 and 16, insome embodiments the user selectable control values are displayed as ifarranged on the surface of a rotating wheel with a currently selectedvalue being closest to the user, e.g., in the center and displayed inlarger font, and other selectable values above and below the currentlyselected value being shown using a smaller size as if further away fromthe user. In the illustrated example of FIG. 20 it can be seen that thecurrently selected exposure control value 2006 (e.g., exposure value of“0” in the example) that will appear to be closest to the user is in thecenter and appears more prominent than other displayed exposure values.The user may swipe his finger on the display screen in upward ordownward direction to change a currently selected exposure control valuewith a new currently selected exposure value moving to the position inthe center adjacent the current selection pointer 2004 and thepreviously selected exposure value moving away from the center position.An exposure control indicator 2002 can also been seen on the displayscreen indicating that control values being displayed correspond toexposure settings. Furthermore the exposure control option icon 1804 isalso displayed on the right side. In some embodiments if the user tapsthe exposure control option icon 1804 the display screen shown in FIG.20 switches back to the main control options display screen 1200 of FIG.12.

FIG. 21 illustrates a user interface screen 2100 displayed subsequent tothe selection of the manual control mode option by the user shown inFIG. 10 as an alternative to the user interface screen 1200 of FIG. 12.The user interface screen 2100 includes an orientation grid which allowsthe user to see the preview image superimposed with a 3×3 grid. Such agrid helps a user artistically compose an image where the user seeks tomaintain a certain proportionately with respect to the captured imageallowing the user , for example, to appreciate how the image will appearif divided into thirds along the horizontal an vertical axis. Thisfacilitates the user's ability to maintain symmetry when composing animage. In addition to the grid, additional informational data thatprofessional photographers may find useful is shown and is displayedover the preview image in the FIG. 21 example. In addition to thecontrol options which are the same or similar to those shown anddiscussed with regard to FIG. 12, the user interface screen 2100 furtherincludes a histogram 2102, e.g., of luminance values based on thecontent of the image, and current metadata 2104, 2106, 2108corresponding to camera control setting related information with theinformation corresponding to such things as an exposure setting 2104,shutter speed 2106 and ISO value 2108 respectively. The displayedhistogram 2102 allows for providing a visual alert to the user regardingthe level of exposure affecting different portions of an image shown asthe preview image. By looking at the histogram 2102 the user canunderstand if parts of an image are being overexposed and/orunderexposed and thus take corrective actions accordingly.

FIG. 22 illustrates a user interface screen 2200 displayed subsequent tothe selection of the ISO control option in FIG. 13. As shown a set ofuser selectable ISO control values are displayed to the user on thedisplay screen. Consider that the user wants to change the currentlyselected ISO value 1406. In order to change the currently selected ISOvalue the user may swipe his finger on the screen in upward or downwarddirection. In the example of FIG. 22 it is illustrated that thecurrently selected ISO value 1406 is “640” and the user swipes his/herfinger 704 in the downward direction as indicated by arrow 2202. Thefinger swipe in the downward direction causes the ISO values, which arearranged as if on the surface of a rotating wheel, to move in theanticlockwise direction and wrap around the simulated wheel with a newcurrently selected value moving to the position in the center closer tothe user. Alternative to the swipe method illustrated in the figure,actual up/down buttons can be pressed to move the list of userselectable ISO setting values up or down with the ISO value in thecenter being the currently selected setting value at a given time. Whilethe currently selected control value is shown in the center in variousfigures it could be positioned at another predetermined location on thescreen depending on the embodiment.

FIG. 23 illustrates a subsequent user interface screen 2300 displayedsubsequent to the user's finger swipe in the downward direction on thedisplay screen 2200 of FIG. 22. As can be appreciated from the figure, anew currently selected ISO value 2302, e.g., ISO value of “1000”, movesto the center while the previously selected ISO value “640” moves downand away from the center position as shown. As can be appreciated fromthe figure the new ISO setting of “1000” which is the current ISOsetting is shown larger and more prominently than other ISO settingvalues including the previous setting of “640”. The camera, while beinga digital camera, will simulate the effect of the currently selected ISOsetting value, e.g., in terms of sensitivity to light, when capturingimages.

FIG. 24 illustrates a user interface screen 2400 displayed subsequent tothe selection of the ISO control option in FIG. 13 wherein the userswipes his/her finger 704 in the upward direction as indicated by arrow2402 in order to change the currently selected ISO value 2404 of “640”.The finger swipe in the upward direction causes the ISO values, whichare arranged as if on the surface of a rotating wheel, to move in theclockwise direction and wrap around the simulated wheel with a newcurrently selected value moving to the position in the center closer tothe user.

FIG. 25 illustrates a user interface screen 2500 displayed subsequent tothe user's finger swipe in the upward direction on the display screen2400 of FIG. 24. As can be appreciated from the figure, a new currentlyselected ISO value 2502, e.g., ISO value of “500”, moves to the centerwhile the previously selected ISO value “640” moves up and away from thecenter position as shown.

FIG. 26 illustrates a user interface screen 2600 displayed when a useremploys a spread gesture to enlarge, e.g., zoom in, a preview imagedisplayed in the background. As illustrated in the figure the user mayuse his/her fingers 2610, 2612 (where in this case the thumb is to beconsidered a type of finger) to implement a spreading motion on thedisplay screen 2600. The arrows 2614, 2616 in the opposite directionindicate the respective directions in which the fingers areswiped/spread on the display screen in order to implement a zoom inoperation. In some embodiments when one of a pinching, spreading ordouble tapping actions is detected on the display screen of the camera,a visual slide bar 2602 is presented to the user. The visual slide bar2602 provides a visual notification to the user of the effective focallength associated with the current zoom setting indicated by thelocation of the pointer 2604. In some embodiments the display alsoprovides an indication regarding which focal length camera modules arebeing used at a given point in time by displaying the focal length ofthe actual camera modules, e.g., 35 mm in the example. The visual slidebar 2602 includes camera module focal length indicators 2603, 2604 and2608 (smaller circles) corresponding to focal lengths of physical cameramodules and a pointer 2604 (larger circle). The position of the pointerat a given time indicates the current effective focal length being usedfor the current zoom setting. Higher focal length camera modules, e.g.,70 mm and 150 mm focal length camera modules, are often used when a useris performing a zoom in operation while 35 mm focal length cameramodules may be used when no zooming is being used. The focal lengthindicators 2603, 2604 and 2608 remain fixed on the slider bar 2602 whilethe pointer 2604 slides over the bar 2602 as zoom in or zoom out isperformed in response to a spread or pinch action respectively. In someembodiments as the user performs spreading using fingers to zoom in thepointer 2604 on the slide bar 2602 moves to the right, e.g., towards thehigher focal length indicators. The first camera module focal lengthindicator 2603 corresponds to the 35 mm camera modules and coincideswith the pointer 2604 in the example of FIG. 26 and thus the indicator2603(smaller circle) is not visible in this illustration because ofbeing superimposed by the pointer 2604 (larger circle). The secondcamera module focal length indicator 2606 corresponds to 70 mm cameramodules and the third camera module focal length indicator 2608corresponds to 150 mm camera modules. Depending on the position of thepointer 2604 an effective focal length setting value corresponding tothe current zoom setting is displayed right below the correspondingfocal length indicator. For example in the illustrated example of FIG.26, the pointer 2604 is positioned at the extreme left, e.g., at thestarting point, corresponding to 1× or no zoom on the slide bar 2602.This pointer position corresponding to no zoom corresponds to theeffective focal length of 35 mm. Thus in this example the effectivefocal length value “35 mm” is displayed below the pointer 2604 whichsuperimposes the camera module focal length indicator 2603.

As the zoom setting is changed, e.g., increased or decreased the pointer2604 moves on the slide bar 2602. In some embodiments when the pointer2604 moves away (e.g., in response to change in zoom in or zoom outlevel being changed by the user's spread or pinch actions) from a givencamera module focal length indicator by more than a predetermined amountof distance on the slide bar 2602, the effective focal length settingvalue disappears from the display. For example when the pointer 2604moves away from the position of the first indicator (not visible in FIG.26 example but located right above the position where focal length value“35 mm” is shown) by more than a predetermined amount of distance on theslide bar 2602, the effective focal length setting value of “35 mm”disappears from the display. Similarly when the pointer 2602 movescloser to a given camera module focal length indicator within apredetermined amount of distance on the slide bar 2602, the effectivefocal length setting value corresponding to the given indicator becomesactive and is displayed.

FIG. 27 illustrates a user interface screen 2600 displayed subsequent tothe screen 2500 when the user continues to perform zoom in operation,e.g., by finger spread action. As illustrated in the figure as the zoomin level is increased the pointer 2604 moves away from the first focallength indicator 2603 towards the higher focal length indicators to theright on the slide bar 2602 and as discussed above the effective focallength value “35 mm” which was previously displayed on the displayscreen 2600 disappears since the pointer 2604 moves away from theposition of the first indicator 2603. Instead, as further illustrated inthe figure, the effective focal length value of “700 mm” is displayedsince the pointer 2604 comes within the predetermined distance of thesecond focal length indicator 2606 thereby indicating to the user theeffective focal length of the currently active camera modulescorresponding to the current zoom setting. While a finger spread methodmay be used to implement zoom, in some embodiments a user may simplytouch and drag the pointer 2604 on the slide bar 2602 to increase ordecrease zoom level. Furthermore the user is allowed to touch/tap anypoint on the slide bar 2602 to bring the pointer 2604 directly to thatpoint and the zoom level changes accordingly to correspond to theselected point on the slide bar. The zoom effect can, and in someembodiment is, reflected in the preview image as the user performs zoomoperation, e.g., by finger spread action, pinching action, doubletapping, moving the pointer 2604 on the slider 2602 etc.

FIG. 28 illustrates a user interface screen 2700 displayed subsequent tothe screen 2600 when the user continues to perform zoom in operation,e.g., continuing the finger spread action. As illustrated in the figure,as the zoom in level is increased the pointer 2604 moves further towardsthe higher focal length indicators to the right relative to the positionof the pointer 2604 shown in FIG. 27. Based on the current zoom settingat the given time, the pointer 2604 moves at the “70 mm” focal lengthindicator mark thus superimposing the focal length indicator 2606thereby making it not visible in the example of FIG. 28. In the FIG. 28example the displayed preview image is what would be produced using a 70mm focal length camera module.

FIG. 29 illustrates a user interface screen 2900 displayed subsequent tothe screen 2800 in the case where the user still continues to performzoom in operation, e.g., continuing the finger spread action. Asillustrated FIG. 29, as the zoom level is increased the pointer 2604moves further towards the highest focal length indicator 2608 to theright most point on the slide bar 2602 relative to the position of thepointer 2604 shown earlier in FIG. 28. The pointer 2604 moves at the“150 mm” focal length indicator mark thus superimposing the focal lengthindicator 2608 thereby making it not visible in the example. At thispoint the zoom level is at the highest possible level and the effectivefocal length value of “150 mm” is displayed to allow more sophisticatedor professional photographers to see the effective focal lengthcorresponding to the current zoom level.

FIG. 30A is a first part of a flowchart 3000 illustrating the steps ofan exemplary method of controlling a camera device, in accordance withan exemplary embodiment.

FIG. 30B is a second part of the flowchart 3000 illustrating the stepsof an exemplary method of controlling a camera device in accordance withan exemplary embodiment. FIG. 30, which comprises a combination of FIGS.30A, and 30B, illustrates the steps of the exemplary method ofcontrolling the camera device in accordance with one exemplaryembodiment. The camera device 100/200 which includes a touch sensitivedisplay 102/215 can be used to implement the methods of the presentinvention including the steps discussed with regard to flowchart 3000.

The method starts in step 3002, e.g., with the camera device beingpowered on and initialized. The method proceeds from start step 3002 tostep 3004. In step 3004 which is part of the initialization operationthe camera operation mode is initialized to be a normal mode ofoperation in which the camera display screen displays preview images ofthe environment/objects which comes under the field of view of thecamera. Thus in some embodiments after the camera is turned on thedisplay screen shows the preview images of one or objects generated bythe camera as the camera points to the one or more objects. In variousembodiments the camera display is a touch sensitive display capable ofdetecting and sensing user's touch and swipe. Operation proceeds fromstep 3004 to step 3006. In step 3006 the camera device monitors for userinput, e.g., to detect an input via the touch sensitive display screen.In various embodiments the monitoring is performed on an ongoing basis.

Operation proceeds from step 3006 to step 3008. In some embodiments step3008 is performed in response to the monitoring detecting receipt of auser input. In step 3008 it is determined if a user input indicatingthat the user seeks to switch from a current mode, e.g., normal/previewmode) of operation to another different mode (e.g., control mode) ofoperation, is received. If it is determined that that such a user inputhas been received the operation proceeds from step 3008 to step 3010otherwise the operation proceeds back to step 3006 and the monitoringoperation continues on an ongoing basis. In step 3010 the current modeof the camera is determined to check if the camera is currentlyoperating in the normal mode. If in step 3010 it is determined that thecamera is currently not in the normal mode of operation the operationproceeds from step 3010 to step 3012 otherwise the operation proceeds tostep 3016.

In step 3012, following the determination in step 3010 that the camerais not in the normal mode, the current mode of the camera is changed andset to the normal mode based on the user input desiring a change fromthe current mode to the other mode. Operation proceeds from step 3012 tostep 3014. In step 3014 the camera display is switched from displayingcontrol mode display features to normal mode display features. Forexample in some embodiments in the normal mode the camera display showspreview image/background but with no additional control options (e.g.,as shown in FIG. 6) while in the control mode of operation the controlsetting options are also displayed in addition to the background previewimage, e.g., such as shown in FIGS. 13-20. Thus in step 3014 the normalmode display is presented. In some embodiments step 3014 furtherincludes sub step 3015 where the camera stops displaying control optionson the display screen as part of switching from the control mode tonormal mode. Operation proceeds from step 3014 back to the monitoringstep 3006.

If in step 3010 it is determined that the camera is currently in thenormal mode of operation the operation proceeds from step 3010 to step3016. In step 3016, following the determination in step 3010 that thecamera is currently in the normal mode, the current mode of the camerais changed and set to the control mode based on the user input.Operation proceeds from step 3016 to step 3018. In step 3018 the cameradisplay is switched from displaying normal mode display features tocontrol mode display features. For example in some embodiments in thecontrol mode the camera display shows preview image/background controlsettings. In some embodiments step 3018 further includes sub step 3020where user selectable control options corresponding to differentcontrollable camera settings which were not displayed in the normal modeare displayed. In some embodiments the user selectable control optionsinclude at least one of an ISO setting, a shutter speed setting, anexposure setting or a zoom setting. In some embodiments the userselectable control options include at least two of an ISO setting, ashutter speed setting, an exposure setting or a zoom setting. Operationproceeds from step 3018 to step 3022. In step 3022 user selection of acontrol option, e.g., one of the displayed user selectable controloptions, is detected. Operation proceeds from step 3022 to step 3023. Instep 3023 a set of user selectable values, for the user selected controloption, are displayed following the detection of user's selection of thecontrol option. In some embodiments a currently selected value in thedisplayed user selectable values is displayed at a predeterminedlocation on the screen. In some embodiments the currently selected valuein the displayed user selectable values is displayed in the center inlarger font size compared to other selectable values which are alsodisplayed. Such a display of a set of user selectable values isillustrated in e.g., FIGS. 14, 16 and 20. In some embodiments optionalsteps 3024 and 3025 are performed as part of step 3023. In step 3024contrast level between a color used to display at least one of the userselectable values and at least a portion of an image to be displayed,e.g., as the preview image, with the at least one of the user selectablevalues is determined. In some embodiments step 3025 is further performedas part of step 3023 where the color used to display one or more userselectable values is changed if it is determined that the contrast levelis below a threshold value. In some embodiments step 3025 of changingthe color used to display one or more user selectable values includesstep 3026 of switching from a darker font color, e.g., black, to lighterfont color, e.g., white, or from lighter font color to darker fontcolor. In some embodiments the switch from black font color to whiteoccurs when the image is a dark image. In some embodiments the switchfrom white to black occurs when white is being used to display the oneor more user selectable values and the image is a light image. It shouldbe appreciated that such a switching and changing of font color used todisplay user selectable values makes the displayed user selectablevalues more prominent and perceivable/legible to the user, e.g., whenthe contrast level between the displayed values and preview image islow. For example if portions of the preview image being displayed arelight in the regions where the user selectable values are beingdisplayed on top of the background preview image then it would behelpful to use a darker font for the user selectable values therebymaking them easily legible.

Operation proceeds from step 3023 to step 3027 where a control indicator(e.g., such indicator 1402 of FIG. 14) is displayed next to thecurrently selected value indicating the type of value being controlled,e.g., corresponding to the type of control option selected by the user.Again this can be seen in the exemplary illustrations of FIGS. 14, 16and 20 where the indicators such as “ISO” “Shutter” and “Exposure” aredisplayed alongside a pointer next to the currently selected value.

Operation proceeds from step 3027 to step 3028. In step 3028 a userinput indicating a desire to change the currently selected valuecorresponding to the user selected control option is detected. In someembodiments the user input is a swipe of a screen indicating a userdesired change in an increasing or decreasing value direction. Operationproceeds from step 3028 to step 3034 via connecting node A 3030. In step3034 the set of user selectable values is changed to move a newcurrently selected value to the predetermined location on the screen inresponse to the user input. In some embodiments step 3036 is performedas part of step 3034 where rotation of a wheel is simulated for displaywith the control values being displayed as if on the surface of thewheel with the currently selected value being closest to the user (e.g.,displayed in the center) and values above and below the currentlyselected value being shown using a smaller size as if further away fromthe user. The sequence of illustrations in FIGS. 22 through 25 allow foran easier understanding of this concept discussed above with regard tostep 3034.

Operation proceeds from step 3034 to step 3038. In step 3028 the controloptions are continued to be displayed. In some embodiments while thecontrol options are continued to be displayed for some additional timebut when no user input is detected within the additional time thedisplayed control options are made to disappear from the screen display,e.g., with the assumption that the user no longer wants to changecontrol values and/or other settings, and the preview image is continuedto be displayed.

Operation proceeds from step 3038 to step 3039. In step 3039 additionaluser input is detected. Operation proceeds from step 3039 to step 3040.In step it is determined whether the detected input indicates that theuser seeks to change/control a zoom setting, e.g., change a currentlyused zoom level. If it is determined that the detected input indicatesthat the user seeks to change a zoom level the operation proceeds fromstep 3040 to step 3043 via connecting node D 3041. In some suchembodiments the detected user input is one of a pinch action, a screentouch and finger spread action or a touch/tap on a location on a zoomcontrol bar. Referring to FIG. 30C which shows process steps along thepath of node D 3041. In step 3043 a zoom control bar including a slider,e.g., sliding pointer that can slide of the zoom control bar, indicatinga current zoom setting is displayed on the user interface displayscreen. Such an exemplary zoom control bar 2602 and pointer 2604 isshown in FIGS. 26-29 and discussed above. Operation proceeds from step3043 to step 3045. In step 3045 a position of the slider is changed,e.g., moved along the zoom control bar as shown in FIG. 27, to show usermodification of the zoom setting made in response to detected user zoomcontrol setting input. Operation proceeds from step 3045 to step 3047.In step 3047 a value corresponding to a zoom setting which matches azoom level provided by a hardware module is displayed while the zoomcontrol bar is displayed, zoom levels between those provided by thehardware module being generated by computation processing of one or morecaptured images. Display of such a value corresponding to a zoom settingis shown in FIGS. 26-29 where the focal length value (e.g., 35 mm, 70mm, 150 mm) corresponding to the zoom level is shown. Operation proceedsfrom step 3047 back to step 3006 via connecting node E 3049.

Now returning to step 3040 of FIG. 30B. If in step 3040 it is determinedthat the detected input does not indicate that the user is seeking tochange a zoom level, the operation proceeds to step 3042. In step 3042it is determined whether the detected input indicates that the userseeks to change a currently selected control value, e.g., change acurrently selected control value displayed on the screen. If it isdetermined that the detected input indicates that the user seeks tochange a currently selected control value the operation proceeds fromstep 3042 to step 3034 and the operation proceeds in the mannerdiscussed above. If it is determined that the detected input does notindicate that the user is seeking to change a currently selected controlvalue, the operation proceeds to step 3044 where it is determined if thedetected input corresponds to a different control option than the onefor which user selectable values are currently displayed, e.g., otheruser selectable control option. If it is determined that the detectedinput corresponds to a different control option than the one for whichuser selectable values are currently displayed the operation proceedsfrom step 3044 back to step 3024 via connecting node B 3046 otherwisethe operation proceeds to step 3048. In step 3048 it is determined ifthe detected user input indicates a desire to switch to normal mode ofoperation. If it is determined that the detected user input indicates adesire to switch to normal mode of operation the operation proceeds fromstep 3048 back to step 3015 via connecting node C 3052, otherwise theoperation proceeds to step 3050. In step 3050 the camera performsfunction corresponding to the detected user input, e.g., turn on flash,turn off camera, view saved captured images etc. Operation proceeds fromstep 3050 back to step 3040 and one or more steps are performed based inadditional detected user input.

FIGS. 31-40 are used to facilitate an understanding of the usefulness ofthe exemplary orientation indicator tool (also referred to as the leveltool) feature provided in the camera device in some embodiments. Theorientation indicator tool provides the user with feedback about theorientation of the device. The orientation implies the pitch, roll andyaw of an object, relative to the pull of gravity.

FIG. 31 illustrates a drawing 3100 showing an object 3102, e.g., a cubewith one letter on each side, placed on a table 3104 and an exemplarycamera device 3106 which may be used to capture a image of the object3102. The object 3102 has the letter “B” on the front facing side(looking at the drawing 3100), the letter “A” on the side facing andcloset to the camera 3106 and the letter “C” on the top of the objectcube 3102. The drawing 3100 shows the orientation of the camera 3106relative to the object 3102 and table 3104, e.g., how the camera 3106 isheld to capture an image of the object 3102.

FIG. 32 illustrates a drawing 3200 showing the camera 3106 and adisplayed preview image 3202 on the camera display screen 3108 at a timewhen the user has activated the exemplary orientation indicator tool,e.g., by selecting the OI option icon 1806. The displayed preview image3202 includes a rendered images 3102′ and 3104′ of the object 3102 andthe table 3104 respectively as it appears to the user viewing the object3102 on the table 3104 through the camera 3106 in the scenario shown inFIG. 31 and an orientation indicator 3204 which represents an object,rendered in 3D space and animated, so as to appear to remain at a fixedorientation relative to the ground and this the horizontal table surfacewhich is parallel to the ground. In the scenario illustrated in FIG. 31since the camera 3106 is in a vertical position with the front portionof the camera 3106 extending in the horizontal direction parallel to theground and horizontal surface of the table 3104 the orientationindicator 3204 in FIG. 32 appears as a generally flat line extendingparallel to the table surface superimposed on the preview image 3102′ ofthe object 3102.

The orientation sensor, e.g., gyroscope and/or accelerometer, output isused to determine camera orientation based on sensor reading andcontrols the position and shape of the orientation indicator 3204 toprovide the user information with respect to camera orientation the realworld.

In some embodiments the shape used as the orientation indicator object3204 is a square plane segment, rendered as an outline, but any shape,object or rendering style is possible, e.g., shape in the form of arectangle in some embodiments. As illustrated the shape is superimposedover the image which is being displayed to provide the user inindication of the horizontal, e.g., ground, position in the real worldeven though the object may be captured at an angle or other orientation.Output from accelerometers, gyroscopes and/or other sensors used todetermine orientation of the camera as images are captured and displayedare used to control the orientation of the orientation indicator objecton the display. Thus, camera tilt and rotation can be detected andreflected in displayed version of the orientation indicator.

The typical fixed orientations are horizontal (landscape/level) andvertical (portrait/plumb), but any preset or programmable orientationsare possible. The level tool provides additional feedback when thedevice is close to one of the target orientations. For example, thetypical indication is a change in color or line thickness of the widget.The widget may change in other ways, such as changing shape. The usercan program one or more desired target orientations and thus theorientation indicator 3204 can be used by the user to determine when adesired target orientation is achieved. This facilitates the user'sability to reliably capture multiple shots using the same orientation atdifferent times and/or at different locations even though it might bedifficult for the user without the aid of the indicator 3204 todetermine a true level condition because of being a hill or otherinclined surface. The typical target orientations are horizontal(landscape/level) and vertical (portrait/plumb), but any preset orprogrammable orientations are possible. The level tool may, and in someembodiments does, provide other indications that the device has reacheda target orientation, such as sounds or vibration. The level tool thusallows the user to accurately position the camera to capture scenes at apredefined orientation. This is particularly important for architectureand landscape photography. The level tool may be selected from byselecting the “more options” control icon, e.g., icon 1006 and/or 1208,and then selecting “orientation indicator” to enable the orientationindicator tool to be displayed.

FIG. 33 illustrates a drawing 3300 showing the object 3102 on the table3104 and the exemplary camera device 3106 held at an angle (indicated byreference 3302) with respect to the horizontal surface of the table 3104which is parallel to the ground. It can be seen that the cameraorientation relative to the object 3102 and table 3104 shown in drawing3300 is different than that shown in drawing 3100. It can be appreciatedfrom FIG. 33 that with the camera 3106 being held in the manner shown inthe drawing, i.e., with the front portion of the camera 3106 extendingin a direction which is at an angle with respect to the ground andhorizontal surface of the table 3104, more than one side of the objectcube 3102 is visible to the user through the camera 3106.

FIG. 34 illustrates a drawing 3400 showing the camera 3106 and adisplayed preview image 3402 on the camera display screen correspondingto the capture scenario illustrated in FIG. 33 and with the orientationindicator tool being activated. The displayed preview image 3402includes rendered images 3404 and 3406 of the object 3102 and the table3104, respectively, as it appears to the user viewing the object 3102 onthe table 3104 through the camera 3106 in the scenario shown in FIG. 33.Also displayed overlaid on the preview object image 3404 is theorientation indicator 3204. As can be seen in the figure, since thecamera 3106 is no longer in a vertical position but rather oriented atan angle with respect to the horizontal, the orientation indicator 3204in FIG. 34 appears as a tilted rectangular outline reflecting the cameratilt and rotation with respect to the ground.

FIG. 35 illustrates a drawing 3500 showing the object 3102 on the table3104 and the exemplary camera device 3106 held directly facing the topface of the object 3102. As can be seen the front portion of the camera3106 extends in the vertical direction downwards and thus about 90degrees (indicated by reference 3302) with respect to the ground andhorizontal surface of the table 3104. It can be appreciated from FIG. 35that with the camera 3106 being held in the manner shown in drawing3500, the top face of the object cube 3102 is visible to the userthrough the camera 3106.

FIG. 36 illustrates a drawing 3600 showing the camera 3106 and adisplayed preview image 3602 on the camera display screen correspondingto the capture scenario illustrated in FIG. 35 and with the orientationindicator tool being activated. The displayed preview image 3602includes a rendered image 3604 of the top face of the object 3102 (facewith letter “C”) as it appears to the user viewing the object 3102through the camera 3106 in the scenario shown in FIG. 35. Theorientation indicator 3204 is also displayed overlaid on the previewobject image 3604 and appears as a rectangular outline but with adifferent perspective compared to the shape and perspective of indicator3204 shown in FIG. 34. In FIG. 36 the orientation indicator is shown asa upright rectangle reflecting the fact that the camera is orientated ina horizontal position facing downward with front of the camera beingparallel to the ground.

FIG. 37 illustrates a drawing 3700 showing the object 3102 on the table3104 and the exemplary camera device 3106 held such that the camera 3106is both tilted, e.g., with respect to the ground or table 3104 surface,and rotated, e.g., anticlockwise with respect to a vertical axisextending from the top face of the object 3102 to the bottom face of theobject 3102 or the table 3104. It can be appreciated from FIG. 37 thatwith the camera 3106 being held in the manner shown in drawing 3700, theobject 3102 can be observed in a different perspective by the userthrough the camera 3106 compared to other drawings.

FIG. 38 illustrates a drawing 3800 showing the camera 3106 and adisplayed preview image 3802 on the camera display screen correspondingto the capture scenario illustrated in FIG. 37 and with the orientationindicator tool being activated. The displayed preview image 3802includes rendered images 3804 and 3806 of the object 3102 (with twosides of the object being visible) and portion of the table 3104respectively as they appear to the user viewing through the camera 3106in the scenario shown in FIG. 37. The orientation indicator 3204 is alsodisplayed overlaid on the preview object image 3804 and appears as arectangular outline oriented in a manner that reflects the camera tiltand rotation with respect to the ground.

FIG. 39 illustrates a drawing 3900 showing the object 3102 on the table3104 and the exemplary camera device 3106 held such that the camera 3106is both tilted, e.g., with respect to the ground or table 3104 surface,and rotated, e.g., clockwise with respect to a vertical axis extendingfrom the top face of the object 3102 to the bottom face of the object3102 or the table 3104. It can be appreciated from FIG. 39 that with thecamera 3106 being held in the manner shown in drawing 3900, the object3102 can be observed in a different perspective by the user through thecamera 3106 compared to other drawings.

FIG. 40 illustrates a drawing 4000 showing the camera 3106 and adisplayed preview image 4002 on the camera display screen correspondingto the capture scenario illustrated in FIG. 39 and with the orientationindicator tool being activated. The displayed preview image 4002includes rendered images 4004 and 4006 of the object 3102 (with twosides of the object being visible) and portion of the table 3104respectively as they appear to the user viewing through the camera 3106in the scenario shown in FIG. 39. The orientation indicator 3204 is alsodisplayed overlaid on the preview object image 4004 and appears as arectangular outline oriented in a manner that reflects the camera tiltand rotation with respect to the ground in such a scenario.

An exemplary method of controlling a camera device, e.g., camera device100/200, in accordance with one exemplary embodiment comprises:detecting user selection of a camera control option; and displaying(e.g., on a touch sensitive display 102 of camera 100) a set of userselectable values for the user selected control option, a currentlyselected value in the displayed user selectable values being displayedat a predetermined location on the screen. In some embodiments thepredetermined location on the screen is a center screen location.

In some embodiments the method further comprises: displaying, prior todetecting user selection of the camera control option, user selectablecontrol options corresponding to different controllable camera settings,said detected user selection of the camera control option being one ofthe displayed user selectable control options. In some embodiments theuser selectable control options include at least one of an ISO setting,a shutter speed setting, an exposure setting or a zoom setting. In someembodiments the user selectable control options include at least two ofan ISO setting, a shutter speed setting, an exposure setting or a zoomsetting.

In some embodiments the method further comprises: detecting user inputindicating a desire to change the currently selected value correspondingfor the user selected control option; and changing said the set of userselectable values to move a new currently selected value to thepredetermined location on the screen in response to the user input. Insome embodiments the user input is a swipe of a screen indicating a userdesired change in an increasing or decreasing value direction. In someembodiments changing the set of user selectable values includessimulating rotation of a wheel with said user selectable values beingdisplayed as if on the surface of the wheel with the currently selectedvalue being closest to the user and user selectable values above andbelow said currently selected value being shown using a smaller size asif further away from the user.

In some embodiments the method further comprises: displaying a controlindicator next to the currently selected value indicating the type ofvalue being controlled (e.g., exposure, film speed, etc). In someembodiments the method further comprises: detecting user inputindicating that the user seeks to switch between a current mode ofoperation to another mode of operation, the current mode of operationbeing one of a normal mode of operation and a control mode of operation,said another mode of operation being different from the current mode ofoperation; and switching between modes of operation in response todetecting user input indicating that the user seeks to switch betweenone of a normal mode of operation and a control mode of operation, saidswitching including displaying one or more control options to a userthat were not being displayed if the switch is from a normal mode ofoperation to a control mode of operation and closing the display of theone or more control options being displayed if the switch is from thecontrol mode of operation to the normal mode of operation.

In some embodiments the method further comprises: determining a contrastlevel between a color used to display at least one of the userselectable values and at least a portion of an image to be displayedwith the at least one of the user selectable values, and if it isdetermined that the contrast level is below a threshold value, changingthe color used to display the at least one user selectable value. Insome embodiments changing the color used to display the at least oneuser selectable value includes switching from black to white or fromwhite to black. In some embodiments switching from black to white occurswhen said image is a dark image. In some embodiments the switch fromwhite to black occurs when white is being used to display the at leastone user selectable value and the image is a light image. In someembodiments the method further comprises: detecting user input (pinch orspread) used to control a zoom setting, displaying a zoom control barincluding a slider indicating a current zoom setting, and changing theposition of the slider to show user modification of the zoom settingmade in response to detected user zoom control setting input. In someembodiments the user input is one of a pinch operation, a screen touchand finger spread operation or touching a location on the slider controlbar. In some embodiments the method further comprises: displaying, whilesaid zoom control bar is displayed, a value corresponding to a zoomsetting which matches a zoom level provided by a hardware module, zoomlevels between those provided by a hardware module being generated bycomputation processing of one or more captured images.

An exemplary camera device, e.g., such as camera 100 or 200, implementedin accordance with the invention, comprises: a touch sensitive displayscreen, e.g., display screen 102/215; and a processor (e.g., processor110/211) configured to detect user selection of a camera control optiondisplayed on the touch sensitive display screen and control the touchsensitive display screen to display a set of user selectable values forthe user selected control option, a currently selected value in thedisplayed user selectable values being displayed at a predeterminedlocation on the screen. In some embodiments the predetermined locationon the screen is a center screen location.

In some embodiments the processor 110/211 is further configured tocontrol the touch sensitive display screen to display, prior to theprocessor 110/211 detecting user selection of the camera control option,user selectable control options corresponding to different controllablecamera settings, the detected user selection of the camera controloption being one of the displayed user selectable control options. Insome embodiments the user selectable control options include at leastone of an ISO setting, a shutter speed setting, an exposure setting or azoom setting.

In some embodiments the processor 110/211 is further configured to:detect user input indicating a desire to change the currently selectedvalue corresponding for the user selected control option; and change theset of user selectable values to move a new currently selected value tothe predetermined location on the screen in response to the user input.In some embodiments the user input is a swipe of a screen indicating auser desired change in an increasing or decreasing value direction.

In some embodiments the processor 110/211 is configured, as part ofbeing configured to change the set of user selectable values, tosimulate rotation of a wheel with the values on displayed as if on thesurface of the wheel with the currently selected value being closest tothe user and values above and below the currently selected value beingshown using a smaller size as if further away from the user. In someembodiments the processor 110/211 is further configured to control thetouch sensitive display screen to display a control indicator next tothe currently selected value indicating the type of value beingcontrolled (e.g., exposure, film speed, etc).

In some embodiments the processor 110/211 is further configured to:detect user input indicating that the user seeks to switch between acurrent mode of operation to another mode of operation, the current modeof operation being one of a normal mode of operation and a control modeof operation, the another mode of operation being different from thecurrent mode of operation; and control the camera device to switchbetween modes of operation in response to detecting user inputindicating that the user seeks to switch between one of a normal mode ofoperation and a control mode of operation, as part of controlling thecamera device to switch between modes the processor 110/211 is furtherconfigured to control the touch sensitive display screen to i) displayone or more control options to a user that were not being displayed ifthe switch is from a normal mode of operation to a control mode ofoperation, and ii) close the display of the one or more control optionsbeing displayed if the switch is from the control mode of operation tothe normal mode of operation.

In some embodiments the processor 110/211 is configured to: determine acontrast level between a color used to display at least one of the userselectable values and at least a portion of an image to be displayedwith the at least one of the user selectable values; and change thecolor used to display the at least one of the user selectable values ifit is determined that the contrast level is below a threshold value.

In some embodiments the processor 110/211 is configured the processor isfurther configured to switch from black color to white color or fromwhite color to black color as part of being configured to change thecolor used to display the at least one of the user selectable values.

In some embodiments the processor 110/211 is configured to: detect userinput (pinch or spread) used to control a zoom setting. In someembodiments the user input is one of a pinch operation, a screen touchand finger spread operation or touching a location on the slider controlbar. In some embodiments the processor 110/211 is configured to controlthe touch sensitive display to display a zoom control bar including aslider indicating a current zoom setting, and control, e.g., change, theposition of the slider to show user modification of the zoom settingmade in response to detected user zoom control setting input. In someembodiments the processor 110/211 is configured to control the touchsensitive display to display, while the zoom control bar is displayed, avalue corresponding to a zoom setting which matches a zoom levelprovided by a hardware module, zoom levels between those provided by ahardware module being generated by computation processing of one or morecaptured images.

An exemplary camera device, e.g., camera 100/200, in some embodimentsincludes a non-transitory computer readable medium, e.g., memory108/213, including computer executable instructions which when executedby a processor (e.g., processor 110/211) control the camera device to:detect user selection of a camera control option displayed on a touchsensitive display screen (e.g., display 102) of the camera device; anddisplay, on the touch sensitive display screen, a set of user selectablevalues for the user selected control option, a currently selected valuein the displayed user selectable values being displayed at apredetermined location on the screen.

In some embodiments the camera device 100/200 includes module forimplementing the functions corresponding to each of the steps offlowchart 3000 discussed above. In some embodiments such modules areimplemented as software modules, e.g., in the memory 108/213. In otherembodiments the modules are implemented in hardware, e.g., as individualcircuits with each module being implemented as a circuit for performingthe function to which the module corresponds. In still other embodimentsthe modules are implemented using a combination of software andhardware.

The modules can, and in some embodiments are, implemented fully inhardware within the processor 110/211, e.g., as individual circuits. Themodules can, and in some embodiments are, implemented fully in hardware,e.g., as individual circuits corresponding to the different modules. Inother embodiments some of the modules are implemented, e.g., ascircuits, within the processor 110/211 with other modules beingimplemented, e.g., as circuits, external to and coupled to the processor110/211. As should be appreciated the level of integration of modules onthe processor and/or with some modules being external to the processormay be one of design choice. Alternatively, rather than beingimplemented as circuits, all or some of the modules may be implementedin software and stored in the memory 108/213 of the camera device, withthe modules controlling operation of the camera device 100/200 toimplement the functions corresponding to the modules when the modulesare executed by a processor, e.g., processor 110/211. In still otherembodiments, various modules are implemented as a combination ofhardware and software, e.g., with another circuit external to theprocessor providing input to the processor 110/211 which then undersoftware control operates to perform a portion of a module's function.

While shown in the FIG. 1 and FIG. 4 embodiments as a single processor,e.g., computer, it should be appreciated that the processor 110/211 maybe implemented as one or more processors, e.g., computers. Whenimplemented in software the modules include code, which when executed bythe processor 110/211, configure the processor 110/211 to implement thefunction corresponding to the module. In embodiments where such modulesare stored in the memory 108/213, the memory 108/213 is a computerprogram product comprising a computer readable medium comprising code,e.g., individual code for each module, for causing at least onecomputer, e.g., processor 110/211, to implement the functions to whichthe modules correspond.

As should be appreciated, the processor 110/211 is configured to controlthe camera device 100/200 or one or more other elements therein, e.g.,such as the touch sensitive display, to perform the functions ofcorresponding steps illustrated and/or described with regard to theflowchart 3000, which are performed as part of the exemplary method ofoperating the camera device.

While the methods have been shown for controlling camera settings suchas ISO film speed, shutter speed, exposure control and zoom level,various other features and/or settings may be controlled by a user inthe same or similar manner. Accordingly, numerous variations on thedescribed and illustrated camera control methods are possible and withinthe scope of the invention. Numerous additional variations andcombinations are possible while remaining within the scope of theinvention.

FIG. 41, comprising the combination of FIG. 41A, FIG. 41B and FIG. 41C,is a flowchart 4100 of an exemplary method in accordance with anexemplary embodiment. In some embodiments, the exemplary method offlowchart 4100 is performed by a camera device, e.g., a cameraapparatus, including multiple optical chains, e.g., a camera deviceshown in any of the FIGS. 1-5. In various embodiments, some of the stepsof the exemplary method of flowchart 4100 are performed by a cameradevice including multiple optical chains, e.g., camera device 100 ofFIG. 1 and FIG. 50, and some of the steps of the exemplary method offlowchart 4100 are performed by a post capture image processing system,e.g., a computer system, e.g., image processing device 5001 of FIG. 50.

Operation starts in step 4102 in which a camera device includingmultiple optical chains is powered on and initialized. Operationproceeds from step 4102 to step 4104, in which the camera devicedisplays a setting2 menu from which a user can select one of at least adepth of field control option, an aspect ratio control option and/or anF setting control option. For example, in one exemplary embodiment, thesetting menu, e.g., settings menu 4202 of FIG. 42, is displayed on thebottom portion of a touch screen of the camera device and the settingmenu includes an icon corresponding to each of a plurality of differentcontrol options. Operation proceeds from step 4104 to step 4106 in whichthe camera device receives user input indicating a user control optionselection, e.g., depth of field control option selection or aspect ratiocontrol option selection. For example, the user selects the depth offield control option by touching the depth of field icon, e.g., depth offield icon 4204 of FIG. 42, on the setting menu, e.g., setting menu, anda sensor in the camera device detects that the touch screen was touchedat the location of the depth of field icon indicating that the depth offield option has been selected. Alternatively, the user selects theaspect ratio control option by touching the aspect ratio icon, e.g.,aspect ratio icon 4206 of FIG. 42, on the setting menu, and a sensor inthe camera device detects that the aspect ratio icon was touched at thelocation of the aspect ratio icon indicating that the aspect ratiooption has been selected.

Operation proceeds from step 4106 to step 4108. In step 4108 the cameradevice determines if the received user input indicates selection of thedepth of field control option. If the determination is the that receiveduser input indicates selection of the depth of field option, thenoperation proceeds from step 4108 to step 4110; otherwise, operationproceeds from step 4108 to step 4112.

Returning to step 4110, in step 4110 the camera device presents the userof the camera device with a user controller depth of field slider, e.g.,depth of field slider 4304 shown in FIG. 43, or other input option forcontrolling a depth of field setting.

In various embodiments, the depth of field slider includes a positionindicator indicating a current depth of field setting position and aplurality of small dots indicating a path over which the positionindicator dot can be moved to a new depth of field setting. For example,FIG. 43, depth of field slider 4304 includes position indicator dot4306, e.g., a large bright dot, and a plurality of small dots, includingsmall dots 4310, . . . , 4312, 4314, . . . , 4316, 4318, indicating apath over which the position indicator dot 4306 can be moved to a newdepth of field setting. Note that the large position indicator dot 4306is as bright as or brighter than small dot 4310; small dot 4312 is notas bright as small dot 4310; small dot 4314 is not as bright as smalldot 4312; small dot 4316 is not as bright as small dot 4314; and smalldot 4318 is not as bright as small dot 4316. In addition small dot 4318is closer to small dot 4316 in comparison to the distance between smalldot 4314 and 4312. Thus small dot brightest and small dot spacing can beare used to identify a path for changing the depth of field setting inthis example. In the example of FIG. 43, the depth of field starts outwith a setting of 1, which is the lowest setting on the slider, and thepath indicator that the position indicator dot can be moved to highersetting values.

In various embodiments, the position indicator is a large dot which isat least as bright as the brightest small dot indicating the path ofwhich the position indicator dot can move. For example, in FIG. 43,position indicator dot 4306 is a large dot and is at least as bright assmall dot 4310 indicating the path in which the position indicator dotcan move. As another example, in FIG. 44, position indicator dot 4406 isa large dot and is at least as bright as small dot 4410 indicating thepath in which the position indicator dot can move. As yet anotherexample, in FIG. 45, position indicator dot 4506 is a large dot and isat least as bright as small dot 4413 indicating a first path in whichthe position indicator dot can move, and is at least as bright as smalldot 4415 indicating a second path in which the position indicator dotcan move.

In some embodiments, the small dots on the path over which the positionindicator dot can be moved are brighter on the side of the slider pathaway from which the slider was most recently moved. For example, in FIG.45, dots on side 4510 are brighter, e.g., an average dot brightnessvalue is higher, than dots on side 4512. Thus, FIG. 45 indicates, basedon a small dot brightness comparison between the two sides of theposition indicator dot, that previously the position indicator was setat a value higher than 8.5, e.g., a value of 10, as shown in FIG. 44.Operation proceeds from step 4110 to step 4114.

In step 4114 the camera device, including multiple optical chains,receives user input indicating a user selectable image control optionsetting. In various embodiments, the received user selectable imagecontrol option setting is not used to control said multiple opticalchains; the received information is stored prior to image capture but isnot used use to control image capture and is use to control post captureimage processing. Step 4114 includes step 4116 in which the cameradevice receives user input to change a depth of field setting from thecurrent depth of field setting. For example, the user swipes along thedepth of field slider from left to right to increase the depth of fieldsetting. In some embodiments, e.g., embodiments, including the exemplarydepth of field slider, operation proceeds from step 4114 to step 4118.In other embodiments, operation proceeds from step 4114 to step 4124.

In step 4118 the camera device changes the location of the depth offield position indicator in response to the received user input.Operation proceeds from step 4118 to step 4120. In step 4120, the camerachecks to determine if the depth of field position indicator has stoppedmoving. In the example of FIG. 43, the position indicator is representedby large dot 4306. If the depth of field position indicator has stoppedmoving, then operation proceeds from step 4120 to step 4122.

In step 4122 the camera device displays a numerical indicator indicatinga new depth of field setting when the position indicator dot is moved toa new location. In various embodiments, the numerical indicator value isnot displayed over the new location while the depth of field positionindicator is moving. In the example of FIG. 44, the numerical indicator4408 displays value 10 above position indicator dot 4406; in the exampleof FIG. 43, the numerical indicator 4308 displays value 1 above positionindicator dot 4306. Note that from FIG. 43 to FIG. 44, the indicator dothas moved from the left of the sliding indicator to the right of thesliding indicator. Operation proceeds from step 4122 to step 4124, inwhich the camera device determines whether or not the user desires achange in the depth of field setting from the current selected depth offiled setting value. If the camera device decides that the user wants achange then operation proceeds from step 4124 to step 4114, where thecamera device receives additional user input. For example, consider thatthe current selected depth of field setting value is 10, as shown inFIG. 44, and that the user wants a lower value. The user swipes thedepth of field slider, e.g., slider 4404 from right to left, causing thedepth of field position indicator, e.g., 4406 as shown in FIG. 44, tomove to a new position, e.g., the position of 4506 as shown in FIG. 45,e.g., performing another iteration of step 4118. Numerical indicatorvalue 4508 indicates 8.5 above position indicator 4506, when theposition indicator stops moving as shown in FIG. 45. Further considerthat the user is satisfied with the user selection of 8.5 for depth offield. Since the user does not desire a change in depth of field fromthe current setting of 8.5, operation proceeds from step 4124, viaconnecting node A 4142 to step 4143.

Returning to step 4112, in step 4112 the camera device determines if theuser input received in step 4106 indicates selection of the aspect ratiocontrol option. If the received user input indicates selection of theaspect ratio control option, then operation proceeds from step 4112 tostep 4126; otherwise, operation proceeds from step 4112 to step 4136.

In step 4126 the camera device presents the user of the camera an inputoption to set an aspect ratio. For example, the camera device presentsthe user of the camera a plurality of user selectable aspect ratios. Insome embodiment, the plurality of user selectable aspect ratios includea 16:9 aspect ratio, a 4:3 aspect ratio and a 1:1 aspect ratio. In somesuch embodiments, the 4:3 aspect ratio corresponds to a maximum imagecapture area, e.g., based on the image sensors being used, and the 16:9and 1:1 aspect ratios corresponds to cropping. In some embodiments, theplurality of user selectable aspect ratios include a 16:9 aspect ratio,a 4:3 aspect ratio, a 1:1 aspect ratio, and a custom aspect ratio. Insome such embodiments, the custom aspect ratio can be set up the user,e.g., via user input to adjust a displayed box corresponding to theaspect ratio which is overlaid over the image, e.g., a cropping box. Invarious embodiments, the custom aspect ratio can be set to a userdesired aspect ratio, e.g., a non-standard aspect ratio, e.g., 2:1,16:8, etc.

In some embodiments, a user selected aspect ratio, which corresponds toa cropped image, corresponds to a predefined location, e.g., a centeredlocation, of the cropped image with respect to the uncropped image,e.g., with respect to the maximum capture area image based on imagesensor.

In various embodiments, in addition to selecting an aspect ratio, theuser can select a positioning of the cropping box corresponding to aselected aspect ratio. For example, the user can move the cropping boxcorresponding to a selected aspect to a desired location and informationis stored specifying the location of the cropping box. For example, thecropping box may be initially placed at a centered position and may bemoved as desired within the boundaries of the overall uncropped image.

Operation proceeds from step 4126 to step 4128. In step 4128 the cameradevice monitors for user input indicating the selected aspect ratio.Step 4128 includes step 4130 in which the camera device includingmultiple optical chains receives user input indicating a user selectableimage control option setting. In various embodiments, the received userselectable image control option setting is not used to control saidmultiple optical chains; the received information is stored prior toimage capture but is not used use to control image capture and is use tocontrol post capture image processing. Step 4130 includes step 4134, andin some embodiments, step 4135. In step 4134 the camera device receivesuser input indicating the selected aspect ratio. In step 4135 the cameradevice further receives user input indicating selected image positioncorresponding to the selected aspect ratio. Operation proceeds from step4128, via connecting node A 4142, to step 4143.

Returning to step 4136, in step 4136 the camera device present the userof the camera another user interface corresponding to another userselectable control option, e.g., a n F-NUMBER setting control option.Other examples, of exemplary user selectable control options which maybe, and sometimes are, presented to a user include, e.g., ISO setting,flash status, and exposure duration. Operation proceeds from step 4136to step 4138. In step 4138 the camera device receives at the cameraincluding multiple optical chains, user input indicating a userselectable control option setting. In various embodiments, the receiveduser selectable image control option setting is not used to control saidmultiple optical chains; the received information is stored prior toimage capture but is not used use to control image capture and is use tocontrol post capture image processing. In some embodiments, step 4138includes step 4140 in which the camera device including multiple opticalchains receives user input indicating the user selected F-numbersetting. Operation proceeds from step 4138, via connecting node A 4142,to step 4143 in which the camera device stores the received userselectable image control option setting, e.g., final selected settingvalue, in a memory of the camera device. Operation proceeds from step4143 to step 4144 or to step 4146. Step 4144, is an optional step whichis performed in some embodiments, for some image control options. Forexample, in one embodiment, step 4144 is omitted if the user selectableimage control option is a depth of field control option; however, step4144 is performed if the user selectable image control option is anaspect ratio control option.

In step 4144 the camera device displays to the user an image, e.g., apreview image with the user selected aspect ratio to which the effect ofthe user selected setting is applied, showing the effect of the receiveduser selectable image control option setting prior to capturing theimages, e.g., in step 4152, used to generate a composite image, e.g., ofstep 4162. For example, consider that the control option is a aspectratio control option, and the user selection is the 16:9 aspect ratiosetting; the camera device display to the user a cropped imagecorresponding to the user selected 16:9 aspect ratio, e.g., which hasbeen cropped from a larger previously display image which represents thelargest area which may be captured which corresponds to the 4:3 aspectratio. Operation proceeds from step 4144 to step 4146. In step 4146 thecamera device determines if the user desires to set another userselectable image control option. If the camera device decides that theuser desires to set another user selectable image control option, thenoperation proceeds from step 4146, via connecting node B 4148 to step4104. However, in step 4146, if the camera device decides that the userdoes not desire to set another user selectable image control option,then operation proceeds from step 4146, via connecting node C 4150, tostep 4152.

In step 4152 the camera device operates multiple optical chains tocapture images after receipt of said user selectable image controloption setting(s). For example, the camera device captures a set ofimage during an image capture time interval, each image in the set ofimages corresponding to a different one of said multiple optical chains.Operation proceeds from step 4152 to step 4154.

In step 4154 the camera device stores in memory one or more capturedimages, e.g., corresponding to the image capture time interval, in afile with metadata including the received user selectable image controloption setting(s) and camera settings. For example, the data, e.g.,pixel values for each set of images captured for a frame time are storedin a file along with metadata indicating the user selectable controloption setting(s), e.g., aspect ratio, depth of field (DOF), F-setting,etc., which are provided at or before image capture, e.g., while theuser is previewing an image of the scene area to be captured. Inaddition, in some embodiments, camera settings are also stored, e.g.,flash status, ISO setting, exposure time, etc. Some of the stored camerasettings may be, and sometimes are based on received user input of aselected option by a user prior to image capture. Some of the storedcamera settings may be, and sometimes are, camera settings derivedautomatically by the camera, e.g., based on a detected sensor input or adetected condition, e.g., low light detection activating a flash, andthe setting may, and sometimes does vary from one frame to anotherwithout user intervention, e.g., when the camera is in an automatic modewith regard to the particular setting parameter. Some of the storedcamera setting may be derived from other camera settings. In variousembodiments, some of the stored camera settings affect image capture.Operation proceeds from step 4154 to step 4156.

In step 4156 the camera devices determines if the user wants to continuecapturing images. If the camera device determines that the user wants tocontinue capturing images, then operation proceeds from step 4156 tostep 4158. In step 4158 the camera device determines if any changes areto be made to user selectable image control option setting forsubsequent image capture. If the camera device determines that changesare to be made to the user selectable image control option settings,then operation proceeds from step 4158, via connecting node B to step4104. However, if the camera device determines that the user does notdesire to make any changes to the user selectable image control optionsettings at this time, then operation proceeds from step 4158 to step4152, in which the camera device operates multiple optical chains tocapture images, e.g. capture a set of images corresponding to anotherimage capture time interval.

Returning to step 4156, in step 4156 if the camera device decides thatthe user does not desire to continue capturing images, then operationproceeds from step 4156 to step 4160 or step 4162, depending upon theparticular embodiment. Step 4160 is an optional step which is performedin some embodiments, in which post capture image processing can be, andsometimes is, performed by a post capture image processing system, e.g.,a computer system external to the camera device which captures theimages being processed. Subsequent steps 4162, 4164, and 4166 areperformed by the camera device or by a post capture image processingsystem. Steps 4162, 4164 and 4166 will be described for an exemplaryembodiment, in which the camera device performs the steps; however, itshould be appreciated that in some embodiments, a post image captureprocessing system which is external to the camera device performs steps4162, 4164 and 4166.

In step 4162, the camera device generates from the images captured bythe multiple optical chains, e.g., during an image capture time intervalbeing currently processed, a composite image in accordance with thestored user selectable image control option setting(s). Operationproceeds from step 4162 to step 4164. In step 4164 the camera deviceoutputs the generated composite image to a display or transmits thegenerated composite image to another device. Operation proceeds fromstep 4164 to step 4166. In step 4166, the camera device determines ifadditional captured image(s) are to be processed corresponding toanother image capture time interval. If additional captured image(s) areto be processed corresponding to another image capture time interval,then operation proceeds from step 4166 to step 4162; otherwise operationproceeds from step 4166 to stop step 4168.

FIG. 42 is a drawing of an exemplary touch screen display view 4200 of acamera device, including multiple optical chains, which may be, andsometimes is, displayed to a user of the camera device, e.g., to presentthe user with user control options, in accordance with an exemplaryembodiment. Display view 4200 includes an image 4201 and a setting menu4202. Settings menu 4202 includes a plurality of different iconsincluding at least some icons corresponding to user control options.Menu 4202 includes a depth of field control option icon 4204 and anaspect ratio control option icon 4206. In some embodiments, there areadditional icons corresponding to additions user control optionfunctions, e.g., an F-number setting control option. In someembodiments, a user of the camera device taps or touches the touchscreen in the location of depth of field control icon 4204 to indicatethat the user has selected the depth of field control option. In someembodiments, a user of the camera device taps or touches the touchscreen in the aspect ratio control icon 4206 to indicate that the userhas selected the aspect control option. In some embodiments, a text typemenu, e.g., a pull down text type setting menu is available for a userto select user control options, e.g., including a depth of field controloption, and an aspect ration control option.

FIG. 43 is a drawing an exemplary touch screen display view 4300 of acamera device, including multiple optical chains, which may be, andsometimes is, presented to a user of the camera device, e.g., to allow auser input a depth of field setting, in accordance with an exemplaryembodiment. Display view 4300 includes an image 4301 and a depth offield (dof) portion 4302. The depth of field portion 4302 includes adepth of field identifier 4303, which is “dof”, for identifying that thepresented controllable slider is for depth of field user input, acontrollable depth of field slider 4304 for controlling a depth of fieldsetting, a depth of field position indicator 4306 for indicating thedepth of field setting. A user slides a finger along the depth of fieldslider 4304 to change the depth of field setting. Depth of fieldnumerical indicator value 4308, located above the current depth of fieldindicator 4306, indicates the current setting of the depth of field whenthe depth of field indicator 4306 is not moving. In various embodiments,the depth of field numerical indicator value is not presented while thedepth of field position indicator 4306 is in motion. In one exemplaryembodiment, the depth of field numerical value is greater than or equalto 1 and less than or equal to 10, with a value of 1 requesting thelowest setting for depth of field and a value of 10 representing thehighest setting for depth of field. In the example of FIG. 43, the depthof field setting is 1. In various embodiments, a setting of 1 is thedefault setting initially presented to a user.

In this example, if the user slides a finger from left to right alongdepth of field slider 4304, the depth of field setting can be increased.

FIG. 44 is a drawing an exemplary touch screen display view 4400 of acamera device, including multiple optical chains, which may be, andsometimes is, presented to a user of the camera device, e.g., to allow auser to input a depth of field setting, in accordance with an exemplaryembodiment. Display view 4400 includes an image 4401 and a depth offield (dof) portion 4402. The depth of field portion 4402 includes adepth of field identifier 4403, which is “dof”, for identifying that thepresented controllable slider is for depth of field user input, acontrollable depth of field slider 4404 for controlling a depth of fieldsetting, a depth of field position indicator 4406 for indicating thedepth of field setting. A user slides a finger along the depth of fieldslider 4404 to change the depth of field setting. Depth of fieldnumerical indicator value 4408, located above the current depth of fieldindicator 4406, indicates the current setting of the depth of field whenthe depth of field indicator 4406 is not moving. In various embodiments,the depth of field numerical indicator value is not presented while thedepth of field position indicator 4406 is in motion. In one exemplaryembodiment, the depth of field numerical value is greater than or equalto 1 and less than or equal to 10, with a value of 1 requesting thelowest setting for depth of field and a value of 10 representing thehighest setting for depth of field. In the example of FIG. 44, the depthof field setting is 10. FIG. 44 may be a representation of the result ofuser input from the starting point of the example of FIG. 43, in whichcase the user has changed depth of field setting from 1 to 10.

With the display as shown in FIG. 44, if the user slides a finger fromright to left along depth of field slider 4404, the depth of fieldsetting can be decreased.

FIG. 45 is a drawing an exemplary touch screen display view 4500 of acamera device, including multiple optical chains, which may be, andsometimes is, presented to a user of the camera device, e.g., to allow auser to input a depth of field setting, in accordance with an exemplaryembodiment. Display view 4500 includes an image 4501 and a depth offield (dof) portion 4502. The depth of field portion 4502 includes adepth of field identifier 4503, which is “dof”, for identifying that thepresented controllable slider is for depth of field user input, acontrollable depth of field slider 4504 for controlling a depth of fieldsetting, a depth of field position indicator 4506 for indicating thedepth of field setting. A user slides a finger along the depth of fieldslider 4504 to change the depth of field setting. Depth of fieldnumerical indicator value 4508, located above the current depth of fieldindicator 4506, indicates the current setting of the depth of field whenthe depth of field indicator 4506 is not moving. In various embodiments,the depth of field numerical indicator value is not presented while thedepth of field position indicator 4406 is in motion. In one exemplaryembodiment, the depth of field numerical value is greater than or equalto 1 and less than or equal to 10, with a value of 1 requesting thelowest setting for depth of field and a value of 10 representing thehighest setting for depth of field. In the example of FIG. 45, the depthof field setting is 8.5, which represents an intermediate point in theselectable range. FIG. 45 may be a representation of the result of userinput from the starting point of the example of FIG. 45, in which casethe user has changed depth of field setting from 10 to 8.5.

With the display as shown in FIG. 45, if the user slides a finger fromright to left along depth of field slider 4504, the depth of fieldsetting can be decreased; and if the user slides a finger from left toright along depth of field slider 4504, the depth of field setting canbe increased. It may be observed that the small dots on the path overwhich the depth of field position indicator doe 4506 can be moved arebrighter on the side 4510 of the slider path matching the direction inwhich the slider was most recently moved than on the other side 4512 ofthe slider path away from which the slider was most recently moved. Thusthe small dot brightness difference between sides 4510 and 4512 ofslider 4504 is used to indicate that previous depth of field setting wasa higher value than 8.5, e.g., was 10.

In some embodiments, the presented image (4301, 4401, 4501) of FIGS. 43,44, and 45 is indicative of the current depth of field setting and apresented image changes in response to user input changes in depth offield setting. In other embodiments, a presented image is not changed toreflect a user input change in depth of field setting.

FIG. 46 is a drawing an exemplary touch screen display view 4600 of acamera device, including multiple optical chains, which may be, andsometimes is, presented to a user of the camera device, e.g., to allow auser to select an aspect ratio setting, in accordance with an exemplaryembodiment. Exemplary display view 4600 includes a captured image 4614,e.g., corresponding to a maximum image capture area, e.g., based on theaspect ratio corresponding to the image sensor. Captured image 4614 isan uncropped image. In this example consider that the image sensorcorresponds to a 4:3 aspect ratio, and thus captured image 4614 has a4:4 aspect ratio. Captured image 4614 includes mountains 4618 and rowhouses 4620. Display view 4606 further includes an aspect ratio userselection portion 4602 for identifying the possible user selectableaspect ratios and for receiving user input. In this example, a user canselect between a 16:9 aspect ratio 4604, as indicated by rectangular boxwith aspect ratio 16:9 and the text “16:9”, a 4:3 aspect ratio 4606, asindicated by rectangular box with aspect ratio 4:3 and the text “4:3” a1:1 aspect ratio 4608, as indicated by a square box and the text “1:1”and a custom aspect ratio 4610, as indicated by the text “custom”. Bytouching or tapping on one of the aspect ratio indicators 4604, 4606,4608 or 4610, the user selects an aspect ratio. If a user selects thecustom aspect ratio, the user can set a non-standard aspect ratio, e.g.,15:9, 4.5: 3, etc. In some embodiments, the user numerically enters thedesired custom aspect ratio. In some embodiments, the user moves acropping box to set the custom aspect ratio.

After a user has selected an aspect ratio, the selected aspect ratio ishighlighted on the screen to indicate that it has been selected. Dashedbox 4612 indicates that the user has selected an aspect ratio of 16:9.In response to the user selection of the 16:9 aspect ratio option,cropping box 4616 is placed over the image 4616 to indicate the imagethat will be visible in the 16:9 aspect ratio. In some embodiments,after user selection of aspect ratio, the area outside the cropping box4616 is darkened or shaded to reduce visibility.

In some embodiments, a cropping box is automatically placed at a centrallocation within the overall image, e.g., with the same amount beingcropped above and below and/or the same amount being cropped left andright.

In some embodiments, in addition to selecting an aspect ratio, a usercan select a location of the cropping box over the overall uncroppedimage. For example, the user may move the cropping box to a desired areaand information indicating the user selected location of the croppingbox is stored in addition to the selected aspect ratio. For example, inFIG. 46 the user may select to move the cropping box to crop off lessarea at the bottom of the image and to crop off more area at the top ofthe image, e.g., in order to capture an image including complete rowhouses 4620, e.g., including the basement potion which would otherwisebe missed if a centralized 16:9 aspect ratio cropping was used.

FIG. 47 is a drawing of an exemplary generated stored file 4700 inaccordance with an exemplary embodiment. File 4700 includes a metadataportion 4702 and a captured images portion 4704. The Metadata 4702includes pre-image capture user selected image control option setting(s)and camera setting(s), e.g., user control manual camera settings and/orautomatic camera settings. In various embodiments, at least some of theuser selected image control option settings have no impact on imagecapture, e.g., with the setting values being used for post capture imageprocessing, e.g., in generating composite images. A user control manualcamera setting is set by the user prior to image capture. An automaticcamera setting can be, and sometimes is, changed by the camera withoutuser intervention, e.g., in response to a detected condition, e.g., aflash may be automatically turned on in response to a detected low lightcondition. Captured images 4704 includes, e.g., sets of captured images,each set of captured images corresponding to a different image capturetime interval, each image in a set of captured images corresponding to adifferent optical chain in the camera device.

In some embodiments, generated stored file 4700 is processed by thecamera device which captured the images, e.g., generating compositeimages. In some embodiments, generated stored file 4700 is processed bya post capture image processing system external to the camera devicewhich captured the images.

FIG. 48 is a drawing 4800 of an exemplary generated stored file 4700′including user selected information, e.g., user selected image controloptions settings and camera settings, and captured images in accordancewith an exemplary embodiment. Metadata portion 4702′ includes framenumber information, represented by column 4802, depth of field setting(DOF) setting information, represented by column 4804, aspect ratiosetting information, represented by column 4806, f-setting information,represented by column 4808, ISO setting information, represented bycolumn 4810, flash status information, represented by column 4812,exposure duration information, represented by column 4813, date taginformation, represented by column 4814, and time tag information,represented by column 4816. Captured images portion 4704′ includes setsof captured images, each set of captured images corresponding to adifferent frame number, each image in a set of captured being capturedby a different optical chain. A captured image includes a set of pixelvalues representing the image. Column 4818 includes images captured byoptical chain 1. Column 4820 includes images captured by optical chain2. Column 4822 includes images captured by optical chain 3. Column 4824includes images captured by optical chain 4. Column 4826 includes imagescaptured by optical chain 5. Column 4828 includes images captured byoptical chain 6. Column 4830 includes images captured by optical chain7.

In the example, of FIG. 48, the camera device receives user input:indicating a setting for depth of field=7, indicating a setting foraspect ratio=16:9, indicating a setting for F setting=F/5.6, indicatinga setting for ISO=400, and indicating a flash setting=OFF, and storesthe received user preference information in memory prior to capturingthe images of frame 1. Some of the received user input settings, e.g.,DOF setting and aspect ratio setting, do not affect image capture, butare used in post capture image processing, e.g., in generating combinedimages in accordance with the user preferences. Some of the receiveduser input camera settings are used to affect image capture. Some of thereceived user input setting are used as part of image capture and arealso used as part of post image capture processing. In this example,exposure duration is calculated by the camera device based on thereceived user input of other parameters.

In the example of FIG. 48, three frames are captured before the userdecides to make a change to a user selected camera setting. During animage capture time interval, corresponding to a frame, seven images arecaptured corresponding to the seven optical chains of the camera device.The captured images of the frame are stored in the file 4700′ along withcorresponding metadata, as represented by a row. For example, the firstrow corresponds to frame number 1 information; the second rowcorresponds to frame number 2 information; and the third row correspondsto frame number 3 information.

In the example, of FIG. 48, after frame 3 image capture, the userdecides to change the user selectable image control option setting fordepth of field and the camera device receives input indicating that thenew depth of field setting is 3, and stored the received input in memoryof the camera device, the new value of depth of field to be included inmetadata in subsequent frames. In this example, a set of images arecaptured for frame 4, and the captured set of images for frame 4 andmetadata for frame 4 are stored in file 4700′; a set of images arecaptured for frame 5, and the captured set of images for frame 5 andmetadata for frame 5 are stored in file 4700′.

In the example, of FIG. 48, after frame 5 image capture, the userdecides to change the user selectable image control option setting fordepth of field, and the camera device receives input indicating that thenew depth of field setting is 6 and stores the received input in memoryof the camera device, the new value of depth of field to be included inmetadata in subsequent frames. The user also decides to change the userselectable image control option setting for aspect ratio, and the cameradevice receives input indicating that the new aspect ratio setting is1:1 and stores the received input in memory of the camera device, thenew value of the aspect ratio setting is to be included in metadata insubsequent frames. In this example, a set of images are captured forframe 6 and the captured set of images for frame 6 and metadata forframe 6 are stored in file 4700′.

Continuing with the example of FIG. 48, after frame 6 image capture, theuser decides to change the user selectable image control option settingfor f-setting, and the camera device receives input indicating that thenew f-setting setting is f/8 and stores the received input in memory ofthe camera device, the new value of the f-setting to be included inmetadata in subsequent frames. The user also decides to change the userselectable ISO setting, and the camera device receives input indicatingthat the new ISO setting is 800 and stores the received input in memoryof the camera device, the new value of the ISO setting is to be includedin metadata in subsequent frames. The user also decides to change theflash status setting, and the camera device receives input indicatingthat the new flash setting is ON and stores the received input in memoryof the camera device, the new value of the flash setting is to beincluded in metadata in subsequent frames. Received user input settingchanges cause the computed value for exposure duration to change from asetting of 5 to a setting of 2, e.g., indicating a shorter exposureduration. In this example, a set of images are captured for frame 7 andthe captured set of images for frame 7 and metadata for frame 7 arestored in file 4700′.

Continuing with the example of FIG. 48, after frame 7 image capture, theuser decides to change the user selectable control option setting forflash, and the camera device receives input indicating that the newflash status setting OFF and stores the received input in memory of thecamera device, the new value of the flash status setting to be includedin metadata in subsequent frames. Received user input setting changescause the computed value for exposure duration to change from a settingof 2 to a setting of 6, e.g., indicating a longer exposure duration. Inthis example, a set of images are captured for frame 8 and the capturedset of images for frame 8 and metadata for frame 8 are stored in file4700′.

FIG. 49, comprising the combination of FIG. 49A and FIG. 49B, is anassembly of module 4900, including Part A 4901 and Part B 4903, whichmay be included in a camera device including multiple optical chains inaccordance with an exemplary embodiment. In some embodiments, some orall of the modules in assembly of modules 4900 are included in a cameradevice and some of the modules, e.g., modules 4962 and 4964, of assemblyof modules 4900 are included in a post capture image processing system,e.g., a computer system external to the camera device which captures theimages being processed. Assembly of modules 4900 implements the steps offlowchart 4100 of FIG. 1 and/or steps and/or features describes withrespect to any of the FIGS. 42-48.

Assembly of modules 4900, which may be included in a camera deviceimplemented in accordance with the present invention, e.g., camera 100of FIG. 1. Assembly of modules 4900 may implement steps of a method,e.g., steps of the method of flowchart 4100 of FIG. 41. In someembodiments, assembly of modules 4900 is an assembly of circuits, whichmay be coupled together. In one exemplary embodiment, assembly ofmodules 4900 is assembly of hardware modules 180 of camera 100 ofFIG. 1. In some embodiments, the assembly of module 4900 is an assemblyof software modules. In one exemplary embodiment, assembly of modules4900 is assembly of modules 118 of memory 108 of camera 100 of FIG. 1.

Assembly of modules 4900 which can, and in some embodiments is, used inthe camera device 100 illustrated in FIG. 1 or another camera deviceincluding a plurality of camera module, e.g., optical chains, e.g.,another camera device including a plurality of camera modules shown ordescribed with respect to any of FIG. 1-50. The modules in the assemblyof modules 4900 can, and in some embodiments are, implemented fully inhardware within the processor 110, e.g., as individual circuits. Themodules in the assembly of modules 4900 can, and in some embodimentsare, implemented fully in hardware within the assembly of modules 180,e.g., as individual circuits corresponding to the different modules. Inother embodiments some of the modules are implemented, e.g., ascircuits, within the processor 110 with other modules being implemented,e.g., as circuits within assembly of modules 180, external to andcoupled to the processor. As should be appreciated the level ofintegration of modules on the processor and/or with some modules beingexternal to the processor may be one of design choice.

Alternatively, rather than being implemented as circuits, all or some ofthe modules in assembly of modules 4900 may be implemented in softwareand stored in the memory 108 of the camera device 100, with the modulescontrolling operation of camera 100 to implement the functionscorresponding to the modules when the modules are executed by aprocessor, e.g., processor 110. In some such embodiments, the assemblyof modules 4900 is included in the memory 108 as assembly of modules118. In still other embodiments, various modules in assembly of modules4900 are implemented as a combination of hardware and software, e.g.,with another circuit external to the processor providing input to theprocessor 110 which then under software control operates to perform aportion of a module's function. While shown in the FIG. 1 embodiment asa single processor, e.g., computer, it should be appreciated that theprocessor 110 may be implemented as one or more processors, e.g.,computers.

When implemented in software the modules include code, which whenexecuted by the processor 110, configure the processor 110 to implementthe function corresponding to the module. In embodiments where theassembly of modules 4900 is stored in the memory 118, the memory 118 isa computer program product comprising a computer readable mediumcomprising code, e.g., individual code for each module, for causing atleast one computer, e.g., processor 110, to implement the functions towhich the modules correspond.

Completely hardware based or completely software based modules may beused. However, it should be appreciated that any combination of softwareand hardware, e.g., circuit implemented modules may be used to implementthe functions. As should be appreciated, the modules illustrated in FIG.49 control and/or configure the camera device 100 or elements thereinsuch as the processor 110, to perform functions of the correspondingsteps illustrated in the method flowchart 4100 of FIG. 41. Thus theassembly of modules 4900 includes various modules that perform functionsof the corresponding steps of the flowchart 4100.

Assembly of modules 4900 includes a settings menu display module 4904, auser control option selection receive module 4906, and a depth of fieldmodule 4910. Settings menu display module 4904 is configured to display,e.g., on a touch screen display included in the camera device, a settingmenu from which a user can select a user control option, e.g., the usercan select one of at least a depth of field control option, an aspectratio control option, and/or an F-number setting control option. Depthof field module 4910 is configured to present the user of the cameradevice a user controllable depth of field slider or other input optionfor controlling a depth of field setting. Depth of field module 4910, insome embodiments, includes a depth of field slider module 4911. Depth offield slider module 4910 is configured to generate and present the userof the camera device with a depth of field slider, the depth of fieldslider presenting the user with a display indicating a current depth offield setting and providing an input interface for changing the depth offield setting, e.g. by sliding a finger across the slider. In variousembodiments, the depth of field slider further displays informationindicating the direction of change from a previous depth of fieldsetting to the currently displayed depth of field setting andinformation indicating the direction or directions in which the slidercan be moved from its current setting. Depth of field slider module 4911includes a position indicator module 4913 and a path module 4915.Position indicator module 4913 is configured to generate and display adepth of field position indicator, e.g., a large bright white dot, onthe depth of field slider to indicate the current setting of the depthof field. Path module 4915 is configured to generate and displayinformation indicates possible path(s) of the depth of field positionindicator, e.g., via small dots on one or both sides of the depth offield indicator. In one embodiment, if there are some small dots on aside of the depth of field position indicator then the positionindicator may be moved in that direction; and if there are no small dotson a side of the depth of field position indicator, the depth of fieldposition indicator is at an extreme setting, e.g., 1 or 10, and can notbe moved any farther in that direction Path module 4915 is configured togenerate and display information indicating a previous path direction ofthe depth of field position indicator, e.g., based on the relativebrightness of dots on each side of the depth of field positionindicator. For example, in one embodiment, small bright dots are on theside from which the depth of field position indicator was previouslymoved from, and a set of decreasing brightness small dots withdecreasing dot spacing are on the other side. Thus if the user wants tocontinue to change the depth of field setting in the same direction asthe last change, the user slides a finger in the direction toward thedecreasing brightness small dots.

Assembly of modules 4900 further includes an aspect ratio setting optionpresentation module 4928, an F-number setting control optionpresentation module 4936, an ISO setting control option presentationmodule 4937, a flash status control option presentation module 4939, andin some embodiments, an exposure duration control option presentationmodule 4941. Aspect ratio setting option presentation module 4937 isconfigured to generate and present the user of the camera device with aninput option to set aspect ratio. In one embodiment, the user isdisplayed with a plurality of user selectable aspect ratios, e.g., usingdimensionally proportionally symbols and/or text corresponding to theaspect ratio options which may be selected, e.g., by touching the symbolor text. F-number setting control option presentation module 4936 isconfigured to generate and present the user of the camera device anF-number setting control input option, e.g. a menu for selecting one ofa plurality of F-settings. ISO setting control option presentationmodule 4937 is configured to generate and present the user of the cameradevice an ISO setting control input option, e.g. a menu for selectingone of a plurality of ISO settings. Flash status control optionpresentation module 4939 is configured to generate and present the userof the camera device an input option for allowing the user to manuallyset the flash to an ON state or to an OFF state. Exposure durationcontrol option presentation module 4941 is configured to generate andpresent the user of the camera device an exposure control input option,e.g. a menu for allowing a user to select one of a plurality ofpredetermined exposure durations.

Assembly of modules 4900 further includes a user selectable controloption setting receive module 4914. User selectable control optionsetting receive module is configured to receive, at the camera device,user input indicating a user selected control option setting, e.g., auser selected image control option setting and/or a user selected cameracontrol option setting. In some embodiments, some received user selectedcontrol option setting are used to control post image capture processingbut not to control camera image capture operations. In some embodiments,some received user selected control option settings are used to controlcamera image capture operations. In some embodiments, some received userselected control option setting as used for both image capture controland post-capture processing operations.

User selectable control option setting receive module 4914 includes adepth of field setting receive module 4916, an aspect ratio inputmonitoring module 4928, a user selected F-number setting receive module4940, a user selected ISO setting receive module 4943, a user selectedflash status setting receive module 4945, and, in some embodiments, auser selected exposure duration setting receive module 4947. Aspectratio input monitoring module 4928 includes a user selected aspect ratioreceive module 4934 and, in some embodiments, a user selected imageposition receive module 4935.

Depth of field setting receive module 4916 is configured to receive userinput to change the depth of field setting from the current depth offield setting. Aspect ratio input monitoring module 4928 is configuredto monitor for user input indicating a selected aspect ratio. Userselected aspect ratio receive module 4934 is configured to receive atthe camera device user input indicating the selected aspect ratio, e.g.,one or a 16:9 aspect ratio, a 4:3 aspect ratio, a 1:1 aspect ratio, or acustom aspect ratio. Optional user selected image position receivemodule 4935, included in some embodiments, is configured to receive userinput indicating a user desired image position of a cropped imagecorresponding to the selected aspect ratio. User selected F-numbersetting receive module 4940 is configured to receive user inputindicating the user selected F-number setting, e.g., one of: f/5.6, f/8,f/11, f/16, f/22, f/32, f/64. User selected ISO setting receive module4943 is configured to receive user input indicating the user selectedISO setting, e.g., one of: ISO 100, ISO 200, ISO 400, ISO 800, ISO 1600,ISO 3200. User selected flash status setting receive module 4945 isconfigured to receive user input indicating whether the user desires tohave the flash on for image captures or to have the flash off duringimage captures. User selected exposure duration setting module 4947,included in some embodiments, is configured to receive user inputindicating a user selected exposure duration setting.

In some embodiments, assembly of modules 4900 includes an exposuredetermination module 4949 configured to determine an exposuredetermination based on one or more received user selected controlsettings, e.g., including an user selected ISO setting.

Assembly of modules 4900 further includes a depth of field positionindicator location change module 4118, a depth of field positionindicator motion detection module 4120, and a depth of field numericalindicator value display module 4122. Depth of field position indicatorlocation change module 4118 is configured to change the location of thedepth of field indicator, e.g., large bright white dot on the depth offield slider, in response to received user input, e.g., in response to adetected swipe motion along the slider. Depth of field positionindicator location change module 4118 is configured to determine if thedepth of field position indicator has stopped moving and to controloperation as a function of the determination, e.g., activate the depthof field numerical indicator value display module 4122. Depth of fieldnumerical indicator value display module 4122 is configured to display anumerical value indicating a new depth of field setting when the depthof field position indicator, e.g. large bright dot, is moved to a newlocation on the depth of field slider, said numerical depth of fieldvalue being displayed over the depth of field indicator. In variousembodiments, the depth of field numerical value is not displayed whilethe depth of field indicator is in motion.

Assembly of modules 4900 further includes a user selectable controloption setting storage module 4942, a control option setting previewmodule 4944, an image capture control module 4952, and a captured imagesfile generation module 4954. Captured images file generation module 4954includes a captured image set storage module 4955 and a metadata storagemodule 4957. User selectable control option setting storage module 4942is configured to store a received user selected control option setting,e.g., a received user selected image control option setting and/or areceived user selected camera option setting, in memory of the cameradevice. Control option setting preview module 4944 is configured todisplay to the user an image, e.g., a preview image showing the effectof the received user selected image control option setting. For example,a preview image with the user selected aspect ratio is displayed to theuser prior to the camera capturing a set of image which are subsequentlyused to generate a composite image. Image capture control module 4952 isconfigured to control the camera device, e.g. control the optical chainsof the camera device, to capture images after receipt of one or moreuser selectable control option setting(s). The user selectable controloption settings include image control option settings and/or cameradevice control option settings. Image capture control module 4952 isconfigured to control the camera device to capture a set of imagesduring an image capture time interval, each image in the set of imagescorresponding to a different one of the optical chains in the cameradevice. Captured images file generation module 4954 is configured tostored in memory one or more captured image, e.g., corresponding to animage capture time interval, in a file with metadata including receiveduser selectable control option settings(s) and camera setting values.FIG. 48 illustrates an exemplary captured images file 4700′ which may begenerated, e.g., row corresponding to one storage iterationcorresponding to an image capture time interval. Captured image setstorage module 4955 is configured to store in the file a set of capturedimages corresponding to an image capture time interval, e.g. onecaptured image per optical chain. Metadata storage module 4959 isconfigured to store a set of metadata, including previously receiveduser selected control option setting(s), the set of metadatacorresponding to the set of captured images, corresponding to the imagecapture time interval.

Assembly of modules further includes a continue capture determinationmodule 4956, a user selectable option change determination module 4958,and a file transfer module 4960. Continue capture determination module4956 is configured to determine if the user of the camera device desiresto continue capturing images or exit image capture and proceed to filetransfer and/or file processing, and to control operation as a functionof the determination. User selectable option change determination module4958 is configured to determine if the user of the camera device desiresto make any changes to the user selectable control option settings, e.g.image control option settings and/or camera control option settings forsubsequent image capture and controls operation as a function of thedetermination, e.g., presents a menu to allow a user selectable controloption to be changed when module 4958 detects that the user desires tomake a setting change. File transfer module 4960 is configured totransfer the stored file including metadata and captured images, e.g.,file including one or more captured images, user selected image controloption(s) and user selected camera device settings, to a post captureimage processing system, e.g., a computer system external to the cameradevice. The file transfer is by wired and/or wireless communications,e.g., via a wireless transmitter included in the camera device to a basestation, e.g., a WiFi base station coupled, e.g. via the Internet, to animage processing computer system. When file transfer module 4960 isincluded in the camera device, the file transfer module 4960 isconfigured to control a transmitter in the camera device to transmit thestored file including metadata and captured images, e.g., file includingone or more captured images, user selected image control option(s) anduser selected camera device settings to a post capture image processingsystem, e.g., a computer system external to the camera device, or to adevice which is coupled to a post capture image processing system. Whenfile transfer module 4960 is included in an image processing computersystem, the file transfer module 4960 is configured to control areceiver in the image processing computer system to receive the storedfile including metadata and captured images, e.g., file including one ormore captured images, user selected image control option(s) and userselected camera device settings, from a camera device which captured theimages in the file or from another device coupled to the camera device.

Assembly of modules 4900 further includes a composite image generationmodule 4962 and a composite image outputting module 4964. Compositeimage generation module 4962 is configured to generate from the imagescaptured by multiple optical chains, e.g., during an image capture timeinterval, a composite image in accordance with the stored userselectable image control option setting(s). A different composite imageis generated for each image capture time interval. Composite imageoutputting module 4964 is configured to output a generated compositeimage to a display, e.g., a display included in the camera device orimage processing system which generated the composite image, or totransmit said generated composite image to another device, e.g., via awired or wireless transmitter included in the device which generated thecomposite image.

FIG. 50 illustrates an exemplary system 5000 including a camera device100 and an image processing system 5001. Camera device 100 may be thesame or similar to the camera device of FIG. 1 which can be used toreceive user input indicating user control setting(s), capture imagesusing a set of camera modules, e.g., optical chains, generate filesincluding sets of captured images and corresponding metadata includinguser selected control setting(s), transfer generated files to otherdevices, and, in some embodiments, generate composite images, inaccordance with the invention. Image processing system 5001 which canreceive generated files including sets of captured images andcorresponding metadata included user control option setting(s) andprocess the received set of images to generate composite images inaccordance with user control option setting(s), e.g., a pre-capture userselected depth of field setting and a pre-capture user selected aspectratio setting. The image processing device 5001 can be a computer systemwhich is coupled via a wire or wirelessly to the camera device 100. Theimage processing system 5001 includes a display 5002 on which one ormore images, e.g., generated composite images or received capturedimages, can be, and sometimes are, displayed. The display 5002 iscoupled via an I/O interface 5012 and bus 5016 to memory 5008, processor5010, assembly of hardware modules 5080, input device 5006 and wirelessand/or wired interface 5014 through which data and information can bereceived and/or transmitted. To support receipt of files includingcaptured sets of images and corresponding metadata and/or otherinformation the interface 5014 includes a receiver 5044 To support theoutputting of data and information, e.g. the outputting of a generatedcomposite image to another device, the interface 5014 includes atransmitter 5046.

The memory 5008 includes an assembly of modules 5003 and data and/orinformation 5020 which can include a received file including sets ofcaptured images and corresponding metadata, captured images, metadata,and/or one or more composite images which may be generated in accordancewith the invention.

The assembly of modules 5003 may include some of the modules of 4900shown in FIG. 49, e.g., modules 4960, 4962 and 4964, with the modulesbeing used, when executed by processor 5010 to control the operation ofthe image processing device 5001. When implemented in hardware theassembly of hardware module 5080 may include some of the modules ofassembly of modules 4900, e.g., modules 4960, 4962 and 4964, but withthe modules being implemented fully in hardware in some embodiments.Depending on the embodiment some modules may be implemented in hardwareand other modules in software.

In some embodiment the camera 100 includes multiple camera modules,e.g., camera modules of a first focal length, a second focal length anda third focal length with each of the camera modules of the differentfocal lengths capturing different size scene areas. In some embodimentsthe receiving of user selected control setting(s), image capture, filegeneration, composite image generation and composite image outputtingmethod is implemented fully in the camera 100, e.g., a handheld camerain which case image processing device 5001 need not be used.

In other embodiments the methods are implemented using the system 5000which includes the combination of the image processing device 5001 andcamera device 100.

The techniques of the present invention may be implemented usingsoftware, hardware and/or a combination of software and hardware. Thepresent invention is directed to apparatus, e.g., dedicated cameradevices, cell phones, and/or other devices which include one or morecameras or camera modules. It is also directed to methods, e.g., methodof controlling and/or operating cameras, devices including a camera,camera modules, etc. in accordance with the present invention. Thepresent invention is also directed to machine readable medium, e.g.,ROM, RAM, CDs, hard discs, etc., which include machine readableinstructions for controlling a machine to implement one or more steps inaccordance with the present invention.

In various embodiments devices described herein are implemented usingone or more modules to perform the steps corresponding to one or moremethods of the present invention, for example, control of image captureand/or combining of images. Thus, in some embodiments various featuresof the present invention are implemented using modules. Such modules maybe implemented using software, hardware or a combination of software andhardware. In the case of hardware implementations embodimentsimplemented in hardware may use circuits as part of or all of a module.Alternatively, modules may be implemented in hardware as a combinationof one or more circuits and optical elements such as lenses and/or otherhardware elements. Thus in at least some embodiments one or moremodules, and sometimes all modules, are implemented completely inhardware. Many of the above described methods or method steps can beimplemented using machine executable instructions, such as software,included in a machine readable medium such as a memory device, e.g.,RAM, floppy disk, etc. to control a machine, e.g., a camera device orgeneral purpose computer with or without additional hardware, toimplement all or portions of the above described methods, e.g., in oneor more nodes. Accordingly, among other things, the present invention isdirected to a machine-readable medium including machine executableinstructions for causing or controlling a machine, e.g., processor andassociated hardware, to perform e.g., one or more, or all of the stepsof the above-described method(s).

While described in the context of cameras, at least some of the methodsand apparatus of the present invention, are applicable to a wide rangeof image captures systems including tablet and cell phone devices whichsupport or provide image capture functionality.

Images captured by the camera devices described herein may be real worldimages useful for documenting conditions on a construction site, at anaccident and/or for preserving personal information whether beinformation about the condition of a house or vehicle.

Captured images and/or composite images may be and sometimes aredisplayed on the camera device or sent to a printer for printing as aphoto or permanent document which can be maintained in a file as part ofa personal or business record.

Numerous additional variations on the methods and apparatus of thepresent invention described above will be apparent to those skilled inthe art in view of the above description of the invention. Suchvariations are to be considered within the scope of the invention. Invarious embodiments the camera devices are implemented as digitalcameras, video cameras, notebook computers, personal data assistants(PDAs), or other portable devices including receiver/transmittercircuits and logic and/or routines, for implementing the methods of thepresent invention and/or for transiting captured images or generatedcomposite images to other devices for storage or display.

Numerous additional variations and combinations are possible whileremaining within the scope of the invention. Cameras implemented in someembodiments have optical chains which do not extend out beyond the frontof the camera during use and which are implemented as portable handheldcameras or devices including cameras. Such devices may and in someembodiments do have a relatively flat front with the outermost lens orclear, e.g., (flat glass or plastic) optical chain covering used tocover the aperture at the front of an optical chain being fixed.However, in other embodiments lenses and/or other elements of an opticalchain may, and sometimes do, extend beyond the face of the cameradevice.

In various embodiments the camera devices are implemented as digitalcameras, video cameras, notebook computers, personal data assistants(PDAs), or other portable devices including receiver/transmittercircuits and logic and/or routines, for implementing the methods of thepresent invention and/or for transiting captured images or generatedcomposite images to other devices for storage or display.

Numerous additional embodiments are possible while staying within thescope of the above discussed features.

What is claimed is:
 1. A method comprising: displaying a settings menufrom which a user can select one of at least a depth of field (DOF)control option and an aspect ratio control option; receiving, at acamera including multiple optical chains, user input indicating a userselectable image control option setting for use in controlling postimage capture composite image generation, said user selectable imagecontrol option setting being a setting that is not used to control saidmultiple optical chains, said user selectable image control optionsetting being one of said depth of field (DOF) control option or saidaspect ratio control option; storing the user selectable image controloption setting in a memory of said camera; and presenting to a user ofthe camera, when the user input indicates selection of the depth offield control option, a user controllable depth of field slider forcontrolling a depth of field setting; wherein said depth of field sliderincludes a position indicator dot indicating a current depth of fieldsetting position and a plurality of small dots indicating a path overwhich the position indicator dot can be moved to a new depth of fieldsetting; and wherein the position indicator dot is a large dot which isat least as bright as the brightest small dot indicating the path overwhich the position indicator dot can move; and operating multipleoptical chains to capture images; and controlling post image capturegeneration of a composite image from the images captured by the multipleoptical chains based on the stored user selectable image control optionsetting.
 2. The method of claim 1, further comprising: displaying anumerical indicator value indicating a new depth of field setting whenthe position indicator dot is moved to a new location on said depth offield slider, said numerical indicator value not being displayed oversaid new location while said depth of field position indicator dot ismoved.
 3. A method comprising: displaying a settings menu from which auser can select one of at least a depth of field (DOF) control optionand an aspect ratio control option; receiving, at a camera includingmultiple optical chains, user input indicating a user selectable imagecontrol option setting for use in controlling post image capturecomposite image generation, said user selectable image control optionsetting being a setting that is not used to control said multipleoptical chains, said user selectable image control option setting beingone of said depth of field (DOF) control option or said aspect ratiocontrol option; storing the user selectable image control option settingin a memory of said camera; presenting to a user of the camera, when theuser input indicates selection of the depth of field control option, auser controllable depth of field slider for controlling a depth of fieldsetting, wherein said depth of field slider includes a positionindicator dot indicating a current depth of field setting position and aplurality of small dots indicating a path over which the positionindicator dot can be moved to a new depth of field setting; displaying anumerical indicator value indicating a new depth of field setting whenthe position indicator dot is moved to a new location on said depth offield slider, said numerical indicator value not being displayed oversaid new location while said depth of field position indicator dot ismoved; and wherein small dots on the path over which the positionindicator dot can be moved are brighter on the side of the slider pathmatching the direction in which the slider was most recently moved thanon the side of the slider path away from which the slider was mostrecently moved; and operating multiple optical chains to capture images;and controlling post image capture generation of a composite image fromthe images captured by the multiple optical chains based on the storeduser selectable image control option setting.
 4. The method of claim 3further comprising: when the user input indicates selection of theaspect ratio control option, presenting to the user of the camera aplurality of user selectable aspect ratios; and monitoring for userinput indicating selection of one of the user selectable aspect ratios.